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GLOSSARY 



ABE 


Adult basic education 


ACE 


Adult and community education 


ALLS 


Adult Literacy and Life-Skills Survey 


ALAF 


Adult Literacy Achievement Framework (draft) 


AR 


At risk learners 


BSA 


Basic Skills Agency (UK) 


CAI 


Computer Aided Instruction 


CASAS 


Comprehensive adult student assessment system 


CTs 


Controlled Trials (also known as quasi-experimental 
research): uses treatment and control groups, but the 
participants are not randomly assigned and may have 
unseen differences; statistical controls allow 
researchers to compensate for the differences between 
the two groups 


Effect size 


A way of quantifying the effectiveness of a particular 
intervention, relative to some comparison intervention. 


Empirical research 


Based on valid and reliable data (rather than theory or 
opinion); data may be quantitative or qualitative 


ESOL 


English for Speakers of Other Languages 


GED 


General Education Development qualification 


IALS 


International Adult Literacy Survey 


ICT 


Information communication technology 


LD 


Learning disabilities (including dyslexia) 


LNL 


Literacy, numeracy and language 


MFLP 


Manukau Family Literacy Project 


NCSALL 


National Center for Study of Adult Learning and 
Literacy (USA) 


NRP 


Report on the National Reading Panel: Teaching 
children to read 


NRDC 


National Research and Development Centre for Adult 
Literacy and Numeracy (UK) 


Phonemes 


The smallest unit of sounds in a word 


Phonemic awareness (PA) 


The ability to hear, differentiate and attend to the 
individual sounds in a word 


Phonics 


The relationship between sounds and the letters that 
represent them 


PAC 


Parent and child together time (a component of the 
Keegan model for family literacy) 
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RCT 



TABE 

TALS 

Tutors 

Tuition 

Wananga 

Whanau 



Randomised controlled Trials (also known as 
experimental research designs) using two identical 
groups of participants randomly assigned to treatment 
and control groups 

Test of Adult Basic Education (a US assessment tool) 
Test of Applied Literacy Skills (a US assessment tool) 
Adult literacy teachers, who may be paid or voluntary 
Teaching or instruction 

A public tertiary institution that provides programmes 
with an emphasis on the application of knowledge 
regarding Maori traditions and custom 

Family 



Literature review of best available evidence on LNL teaching 




EXECUTIVE SUMMARY 



Project brief and background 

The purpose of this literature review is to provide a critical evaluation of the available 
research evidence about effective practices in literacy, numeracy and language 
(LNL) teaching and programme provision in order to inform policy development within 
the broader arena of foundation learning. 

This review has sought original research studies that reliably relate specific aspects 
of teaching practice and programmes’ operations to learning outcomes - especially 
demonstrable changes in the literacy skills of learners. Over the last three decades 
there has been a considerable amount of research and writing in the sector, most of 
which did not meet the criteria for this review. Evaluation studies are by far the most 
common type of research. Their purpose is usually to illustrate programme benefit 
and learner gain but they rarely supply sufficient evidence of learner gain or evidence 
of the specific factors that may have lead to it, to warrant inclusion in this review. 
There is also a significant body of literature that discusses approaches to teaching 
and programme organisation that appear to be more driven by philosophy and 
definitions of literacy than by empirical evidence of what works and have therefore 
not been included. 

Despite the growing recognition of the importance of LNL, there is still a dearth of 
specific research relating to this area in New Zealand and the situation is only 
marginally better overseas. It is noteworthy however that in the process of 
undertaking this review, we have become aware of a considerable number of 
substantial, intervention studies that are currently underway or due for completion in 
2005-2006. The results of these studies are likely to be very useful to us in future. 



Methodology 

Two experienced database researchers initially undertook a systematic search of 
more than 60 databases and research web pages using a combination of key words 
derived from the project brief. The intent was to locate high quality studies (either 
quantitative or qualitative) that demonstrated a clear relationship between learner 
outcomes and specific components of teaching or provision. All potential studies (in 
excess of 500) were initially read where possible (usually on-line) and screened for 
relevance to the project. More than 300 studies broadly matching the project brief 
were then located and acquired through downloads, library interloan or direct access. 
Each of these articles was then read by at least two of the research team and rated 
for its validity in terms of the project brief. Studies that satisfied both reviewers were 
included in the review. 

Initially we were seeking experimental research studies. 1 However, the overall quality 
of the studies located was not particularly high and fell somewhat short of 
comparable reviews carried out in schooling and the early childhood sectors. On the 



1 Either randomised controlled trials, or controlled trials, where groups of learners received different 
interventions and any effects were statistically measured. 
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advice of the project’s technical advisor Professor John Hattie, the scope of the 
research was expanded to include case studies, observational studies and 
collections of professional wisdom from practitioners, in order to ensure a reasonable 
pool of studies to consider and to identify ‘the best available evidence’. 



Findings 

The findings of this review need to be considered tentative, due to the limitations of 
the research base from which they are drawn. The difficulties of researching adult 
LNL have already been described; research to date has been of variable quality and 
much of it has involved such small sample size that it is difficult to generalise from 
the findings. Therefore, we cannot present a list of factors that will definitively ensure 
effective teaching of adult literacy, numeracy and language learners and result in 
learners’ gaining LNL skills. 

However, an analysis across the strongest studies and reviews found some 
congruency of findings, which enables us to group those findings and have some 
confidence that the particular factors identified are likely to contribute to learner gain. 

Factors that appear likely to enhance learner gain include: 

> Appropriately skilled teachers who can identify the strengths and weaknesses 
learners have in speaking, reading, writing and numeracy. 2 Findings from two 
studies suggest that full-time teachers are more likely to enhance learner 
gain, and that learners benefit when there is assistance from teacher aides or 
volunteer tutors. Findings also suggest the importance of teachers having 
adequate non-teaching time for planning and professional development 
(Basic Skills Agency, 2000; Benseman, 2001; G Brooks et al., 2001; 
Fitzgerald & Young, 1997; Kruidenier, 2002b; Padak, Sapin, & Baycich, 
2002 ). 

> Deliberate and sustained acts of teaching, clearly focused on learners’ 
diagnosed needs. All LNL learners, including those who have learning 
disabilities or dyslexia, would benefit from teachers who are able to offer a 
range of teaching strategies. Many studies commented that the actual amount 
of deliberate teaching in LNL programmes was often not very high. (Basic 
Skills Agency, 2000; Condelli, 2003; Condelli & Wrigley, 2004a; Kruidenier, 
2002b; Rice & Brooks, 2004; H.S Wrigley, 2003) 

> A curriculum that is linked to the authentic literacy events that learners 
experience in their lives. An authentic curriculum appears to lead to gain for 
learners in family literacy, for ESOL learners and for mainstream LNL learners 
(supported by more tentative findings that an authentic curriculum assists in 
achieving learner gain in workplace and prison programmes). (Condelli, 2003; 
Condelli & Wrigley, 2004a; Kruidenier, 2002b; Padak & Padak, 1991; Purcell- 
Gates, Degener, Jacobson, & Soler, 2002) 

> Programmes that allow for high levels of participation, probably more than 
100 hours of tuition. Learners with low levels of skill need more tuition for 



2 This finding resonates with the research on effectiveness in New Zealand schools (Hattie, 2002a), 
where “what teachers know, do and care about” is recognised as one of the greatest influences on 
student learning (p. 7). 
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longer, as do ESOL learners. It should be noted that the research reviewed 
had a mix of findings. Three studies (including one ESOL study) found 
learners made gain when receiving over 100 hours teaching (with one study 
suggesting learners would need 300+ hours to move between levels). Two 
other findings suggested fewer hours might also be effective; one of these 
found learners improved when learners had more than 50 hours provision; 
while in another the greatest level of improvement occurred when learners 
had 51-60 hours between pre- and post-tests. Researchers commented that, 
regardless of the exact number of hours before learners made gains, for 
many LNL participants the total hours of teaching received were likely to be 
considerably less than the figures suggested above. While most of the 
research focused on total amount of provision, the intensity or regularity of 
tuition is probably also important. For example, one study suggested that 
learners made less gain once they received more than nine hours teaching 
per week. Intensive courses may be particularly important. For some ESOL 
learners, regular attendance was a more significant variable in achieving skills 
than the actual hours per week. (Basic Skills Agency, 2000; Boudett & 
Friedlander, 1997; Comings, 2003; Kruidenier, 2002b; Shameem, McDermott, 
Blaker, & Carryer, 2002) 

> Explicit teaching of reading, by teachers who are well trained in the reading 
process, and who are skilled in identifying reading difficulties and using 
appropriate teaching strategies to address them. Findings suggested learners 
are more likely to make gain when there is explicit, structured teaching of 
alphabetics, fluency, vocabulary building and comprehension. ESOL learners 
gain when there is explicit teaching of comprehension and vocabulary. 
Reciprocal reading was identified as an effective teaching strategy; others 
what warrant further investigation include oral reading, strategies to increase 
comprehension and fluency-related strategies. (Besser et al., 2004; 
Kruidenier, 2002b; Rich & Shepherd, 1993; Sabatini, 2002; Snow & Strucker, 
2000 ) 

> On-going assessment that takes into account the variation in learners’ skills 
across the dimensions of reading and writing. Assessment processes need to 
incorporate measurement of all four components of reading: alphabetics, 
fluency, vocabulary and comprehension. The assessment of reading needs to 
use more than silent reading and oral comprehension questions as 
assessment tools. A study of learners’ perspectives also identified the 
importance of on-going assessment. (Besser et al., 2004; Davidson & 
Strucker, 2002; Kruidenier, 2002b; Ward, 2003) 

> ESOL programmes structured to maximise oral communication, discussion 
and group work. Findings suggest learners make gain, particularly in 
comprehension, when there are bi-lingual tutors who teach concepts and 
explain learning tasks and instructions using learners’ first languages and 
when they attend regularly. (Condelli & Wrigley, 2004; Condelli & Wrigley, 
2004a; C. Roberts et al., 2004; Shameem et al., 2002) 

> Programmes that deliver clearly structured teaching using a range of 
methods. Provision needs to occur in a range of contexts that: meets 
learners’ needs; that allows for learning plans for every learner; and where 
those plans are related to regular assessments and reviews with learners. 
Most of these factors are not necessarily unique to LNL teaching - they 
should be components of any quality adult education provision. (Basic Skills 
Agency, 2000; Boudett & Friedlander, 1997; G Brooks et al., 2001 ; G. Brooks 
et al., 2001; Comings, Parrella, & Soricone, 1999, 2000; Condelli & Wrigley, 
2004a; Eldred, 2002; Yaffe & Williams, 1998) 
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> Writing programmes that use writing based on expressing learners’ 
experiences and opinions. Programmes that include project-based instruction 
that focus on issues of common interest, on authentic tasks and materials and 
that encourage a variety of writing activities are more likely to promote gain. 
(Gillespie, 2001; Kelly, Soundranayagam, & Grief, 2004; Purcell-Gates et al., 
2002 ) 

> Making efforts to retain learners, including pro-active management of the 
positive and negative forces that help and hinder persistence. Findings 
suggest that childcare, transport assistance, and access to social services 
make a difference, as does building self-efficacy and self-confidence in 
learners and ensuring that learners receive personalised attention. (Comings 
et al., 1999; Eldred, 2002; B. A. Quigley & Uhland, 2000; Yaffe & Williams, 
1998) 

> Family literacy programmes that have a clear focus on literacy/numeracy 
development: Findings suggested that programmes that are more likely to 
ensure gain have parents committed to improving their children’s chances, 
have teaching sessions for parents only and children only, as well as 
together; have home visits; collaborate with other participating groups, to 
ensure programme and services integration; and have staff whose skills 
match the unique challenges of family literacy. (Alamprese, 2001b; 
Benseman, 2002, 2003c, 2004; Padak, Rasinski, & Mraz, 2002; Philliber, 
Spillman, & King, 1996; St Pierre et al., 2003; St. Pierre et al., 1995) 

There are some factors that may enhance learner gain, but for which there has been 
only limited research and where findings are even more tentative: 

> Programmes that have an awareness and management of critical periods in 
the programme (when learners were susceptible to failure and/or withdrawal); 
provide pastoral care in times of need and crisis; have adequate fiscal 
resources and facilities; have administrative leadership. 

> Teaching staff who undertake regular professional development; praise and 
encourage learners; are open as people to their students and have both 
credibility in their field and a passion for their subject. 

> Assessment that includes self-assessment by learners, and constructive, 
verbal feedback from the tutor. 

> Teaching that creates a positive and supportive learning environment; is able 
to balance challenge and support for learners; encourages peer support in the 
form of sharing skills and ideas and friendships; accentuates learners’ 
strengths. Again, these attributes appear to be aspects of good adult 
education practice. 

> Computer Assisted Instruction may be a fruitful teaching strategy in teaching 
mathematics and other aspects of LNL in comparison with ‘traditional’ 
teaching and with learners with low levels of skill; does not replace skilled 
teachers; may motivate learners and develop self-confidence. 

> Reading programmes that ensure learners’ prior knowledge is used from the 
beginning (in integrated programmes), rather than assuming that adult 
beginning readers need to concentrate on strategies for processing print first; 
that include various strategies related to teaching alphabetics, fluency, 
vocabulary and comprehension as suggested in K-12 research; that teach 
learners to monitor their comprehension and understanding as they read; and 
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that are at least 70% on task (i.e. that the majority of time is spent on reading 
and writing tasks). 

> Family literacy programmes that: have public celebrations of key events and 
achievements and that have a ‘second step’ programme for the adult learners 
as they move on from the programme to maintain the “community of 
learning”. 

> Community-based programmes that: have community ownership of programs 
and resources and where there is trust and collaboration between providers 
and communities. 

> Bridging programmes that: are centralised, especially their co-ordination 
(rather than being run in individual academic departments). 

> Workplace literacy programmes where there is commitment from the 
participating company, tuition occurs during work time and where curriculum 
is related to real-life demands. 

> Prison-based programmes that: incorporate a community-building process 
into their reading programme; and where there is contextualised content in 
reading teaching, as this increases attendance rates and levels of 
engagement. 

Fourthly, there are factors that are not supported by research evidence at this time. 

> There are no clear findings that one form of delivery (1 :1 /small group/class) or 
context is better than any other. There were numbers of studies with very 
small numbers of learners saying 1 :1 is effective, as well as others saying that 
participation in small groups is what makes programmes ‘not like school’. 
There was evidence that a range of factors will impact on effectiveness in any 
form of delivery or context e.g. students’ motivations to attend and their life 
stages, the language and literacy diversity of students within a group, 
strategies to retain students, the extent programmes use authentic contexts 
and materials, the intensity of instruction and the skill of the teachers. 

> There is no evidence to support a policy of differentiating dyslexic from non- 
dyslexic students in adult literacy, numeracy and ESOL. 

> The research evidence on numeracy does not provide guidance on the time 
and the nature and extent of teaching required for adults to make significant 
progress. 

Finally, we were not able to identify any research that met the criteria for our study on 
the following subjects listed in the research brief. 

> integrated or embedded provision 

> any socio-demographic characteristics of tutors 

> strategies related to teaching te reo Maori or Maori programme provision 

> factors associated with progress in numeracy 

> assessment and its effect on learning outcomes. 
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Recommendations 

The methodologies of any New Zealand studies need to be enhanced in a number of 
ways: 

> Ensuring that all studies (even if when there are problems with methodologies 
or findings) are released into the public arena: without critical review of the 
work being done, the foundation learning field cannot mature and develop. 

> Learning from overseas by tracking the large scale studies being developed 
by specialised LNL research organisations and replicating appropriate studies 
so we can build systematically on a larger body of knowledge. 

> Reducing the number of small studies and maximising learner numbers in any 
projects undertaken so that findings have the potential to be more 
generalisable. Small-scale intensive insight studies may be more appropriate 
for the New Zealand context and scale of provision than larger controlled 
trials but nonetheless, we need to endeavour to achieve reasonable sample 
sizes. 

> Ensuring systematic literature reviews are conducted for all research projects 
and the findings integrated into final reports. 

> Efforts are made to disseminate findings to practitioners in a number of ways 
including, but not restricted to print publications. 



A research programme could usefully be developed over the medium term in a 
number of areas, including (but not limited to): 

1. LNL teachers: this review has showed how central LNL teachers are in 
achieving learner outcomes and yet we know remarkably little about who 
these people are, the contexts and nature of their employment and the 
organisation of their work, their motivations and aspirations, training, 
issues and philosophies of teaching and the impact of professional 
development they undertake etc. A large-scale survey would provide 
useful baseline information for the future expansion of both initial and on- 
going professional development programmes. Any study should 
encompass whether tutors work with ESOL learners and/or teach 
numeracy. When investigating numeracy information would be useful on 
what they teach, what their professional development needs are; and their 
beliefs and perceptions of self-efficacy related to their own numeracy. 

2. Specific teaching methods for reading: New Zealand has had a long 
history of research into child-related reading that has been largely 
untapped by the adult LNL sector. There is scope to take some of the 
methods identified as potentially useful in this review and construct 
research to test their effectiveness in New Zealand LNL contexts. Some 
aspects that warrant attention include: reciprocal reading, which the 
research indicates is an effective means for teaching adults, oral reading, 
and strategies to develop fluency and comprehension. 

3. Authentic curriculum: it would be very useful to identify to what extent 
authentic curriculum is being incorporated into teaching across the variety 
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of contexts of LNL provision in New Zealand, and analyse issues around 
its use and the opportunities this approach offers us. 

4. Learner-focused longitudinal research: a longitudinal, intensive study 
of approximately 20-30 learners over two years (with the expectation of 5- 
10 dropping out over that period) could provide information about what 
learners believe they learn, how their LNL skills progress and what they 
believe has helped and hindered them. A case study approach would also 
provide a rich description of additional programme factors, for example, 
effective recruitment, attendance patterns, barriers to participation, and 
learner progression on to other outcomes in work and education. 
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1 INTRODUCTION 



1.1 Background 

New Zealand’s participation in the second round of the International Adult Literacy 
Survey (IALS) in 1996 (OECD, 1997) was an important milestone in the development 
of this educational sector. The survey’s findings showed that a significant proportion 
of the New Zealand adult population had considerable literacy difficulties. The study 
was significant not only for establishing the degree of overall need for literacy 
education among adults, but also for its demonstration that this need was not 
confined to any particular social group - although some groups clearly had greater 
levels of need than others. IALS also underlined the complexity of literacy skills in its 
use of three dimensions (prose, document and quantitative) and the measurement of 
these skills along a continuum, in contrast to the common public perception of a 
literacy/illiteracy dichotomy. 

Since the publication of the IALS findings, foundation learning, (a broader term than 
adult literacy and incorporating literacy, numeracy and language, or LNL) has been 
increasingly recognised as an important component across the tertiary sector 
(Benseman, 2003b; Cain Johnson & Benseman, in press). The importance now 
placed on foundation learning is reflected in its inclusion in the current government 
policies and strategies (Ministry of Education, 2001, 2002, 2003, 2003a). 

There has been a growing interest in the role of evidence-based research to inform 
the development of policy and improve the quality of provision. 



1.2 Project brief 

The purpose of this literature review was to provide a critical evaluation of research 
evidence about effective practices in adult literacy, numeracy and language (LNL) 
teaching and programme provision. 3 This report forms part of a cluster of research 
studies to establish benchmarks in foundation learning provision in Aotearoa/New 
Zealand during 2004. 4 

It is important to stress that this review has sought original research studies 
that relate specific aspects of teaching and/or programmes’ operations to 
learning outcomes - especially demonstrable changes in the literacy skills of 
learners. Over the last three decades there has been a considerable amount of 
research and writing in the sector, but most did not meet the criteria for this review. 
Evaluation studies were by far the most common type of research identified in our 
initial search; typically these evaluations identified the benefits gained from 
programmes without linking outcomes to specific aspects of the programme, and in 



3 This study does not include consideration of other, more diverse, elements sometimes included in the 
concept of foundation learning or foundation skills, such as critical thinking, foreign languages, problem 
solving, team work, and motivation to learn. 

4 The other elements include a project to map the extent and nature of foundation learning provision 
nationally, an initial survey of assessment methodologies and an observation study of teachers of adult 
literacy, numeracy and language. 
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particular, with very little analysis of which specific aspects of curriculum or teaching 
practice had a positive impact on learners’ skills. There is a large amount of research 
that describes and advocates particular teaching models based on application in one 
or two programmes, but without supplying sufficient evidence of learner gain to 
warrant inclusion in this review. There is also a significant body of literature that are 
‘position pieces’, some theorising from existing research, some appearing to be more 
driven by philosophy and definitions of literacy than by empirical evidence of what 
works. These are not research studies per se and have therefore not been included. 



1.3 Using an evidence-based approach 

Evidence-based research first emerged in medicine over 50 years ago and is widely 
acknowledged as the cornerstone to that field’s development of effective protocols for 
the treatment and prevention of disease. The move to using an evidence-based 
approach in education has been more recent, but has gathered momentum over the 
past few years as educators have been required to build programmes on more solid 
evidence about ‘what works’ (a common assertion is that education has been based 
90% on professional ‘wisdom’ and only 10% on empirical evidence 5 ). 

In the US for example, federal legislation such as the 2001 No Child Left Behind Act 
included demands of accountability based on research that meets ‘gold standards’ of 
quality. In 2002 the Education Sciences Reform Act was signed, “to provide for 
improvement of Federal education research, statistics, evaluation, information and 
dissemination.” The Act led to the establishment of the Institute of Education 
Sciences, charged essentially with moving towards a 90% empirical evidence/10% 
professional wisdom split. 

The empirical evidence being sought has been based on a hierarchy of research 
methods: 

Experimental designs using two identical groups of participants randomly 
assigned to treatment and control groups (also known as randomised 
controlled trials or RCTs). 

Quasi-experimental designs also use treatment and control groups, but the 
participants are not randomly assigned and may have unseen differences; 
statistical controls allow researchers to compensate for the differences 
between the two groups (also known as controlled trials or CTs). 

Correlational with statistical controls employs treatment and control groups 
that are not identical, but statistical controls are used to compensate for 
differences that may be important. 

Correlational without statistical controls employs treatment and comparison 
groups that are different, but the differences are assumed to be not important, 
usually because of the large sample size. 

Case studies may employ only a treatment group and assumes that the 
differences among participants are not important or obvious, since the sample 



5 Empirical evidence is based on experiment and observation rather than on theory or opinion. It can be 
both quantitative and qualitative. 
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is usually small (Grover J. Whitehurst, Director of the Institute of Education 
Sciences, quoted in Comings, (2003). 

The value attributed to research evidence is usually related to its location on this 
hierarchy, with greatest recognition of the top categories and historically at least, 
decreasing acknowledgement of the latter categories. 

There has been considerable debate in educational circles about the merits and 
limitations of the increased focus on empirical evidence and it is worth noting that the 
quantitative/qualitative pendulum has swung a number of times over recent years. 
For example, in a review of the current state of quantitative research in reading 
(Kamil, 2004, p. 100) the author notes “two decades ago, experimental quantitative 
methodology dominated reading research. A decade ago, there were many who were 
suggesting the end of experimental quantitative research as a paradigm.” 

Research programmes in adult basic education (ABE) in many western countries 
have also been changing to have more of a focus on empirical research, but not 
without real concerns. Researchers in the LNL field have challenged the desirability 
and even feasibility of restricting the research evidence to such a narrow weighting of 
studies that are simply inappropriate for the complex world of LNL learners and 
programmes. 6 

The diversity of adult education 7 is badly served by the narrowing of research. 
The knowledge base is already too narrow, and focussing resources into one 
fairly limited research paradigm is unlikely to lead to anything than a loose set 
of de-contextualised findings (Belzer & St. Clair, 2004). 

Using randomisation (as in RCTs) as a principal determinant of whether research is 
of good quality may be neither accurate nor useful in the LNL context. In the course 
of this project, Professor John Hattie critiqued a recent review of RCTs and CTs in 
adult literacy, numeracy and language (Torgerson et al., 2004) and the source 
documents for 21 of the studies involved in that review. His comments below 
illustrate how their quality affects the potential value that controlled trials offer in 
identifying the impact of interventions. A full version of his critique is provided in 
Appendix A. 

Moreover, choosing specified designs, no matter how defensible, assumes 
that studies using these designs are necessarily somehow superior to studies 
using other designs. What is striking about the 21 studies we located from 
Torgerson et al. is their variable quality - with most of low quality. Using 
randomised or controlled designs clearly does not lead to high quality studies. 
The median sample size is only 52, and given there are at least two groups 
(experimental and control) the ‘typical’ study has only 26 people in each 
group. The average attrition rate is 66%, so two-thirds of each sample did not 
complete the study. It would have been more defensible to include all 
possible studies, code them for the nature of the experimental design, and for 
the quality of the study and then use meta-analysis techniques to address 
whether the effects differ as a consequence of design and quality. The aim 



6 These concerns are not confined solely to researchers of adult LNL (Kamil, 2004; Lomax, 2004). Even 
in areas such as heart surgery, arguments have been made for the inclusion of practitioner knowledge 
to make empirical findings work better in everyday practice (Berwick, 1996, quoted in Comings, 2003, p. 
8 ). 

7 Adult education is often used synonymously with adult literacy in the US. 
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should be to summarise high quality studies, regardless of their design - if 
quality is a moderator to the final conclusions. 

This collection of articles clearly points to the delimitations in making a 
method of assignment (randomisation) the major consideration in deciding on 
the quality of studies. It is difficult to imagine a more unconvincing collection 
of studies - with low sample size, high attrition, low statistical power, and so 
many pre-test differences. As well, there are so few studies that critical 
moderators cannot be investigated in any meaningfully generalisable way to 
evaluate their potential influences on the effect-sizes 8 . Most important, if more 
studies had been included then the key assumption that the effect-sizes from 
RCT and CT are different according to the design of the study, could have 
been evaluated. 



A more appropriate view of evidence-based practice (de-emphasising the value of 
particular research methodologies) to see it as integration of professional wisdom 
with the best empirical research available (Whitehurst, 2004). This broader 
interpretation enables the findings from research based on rigorous methodology to 
be combined with action research, observations and analyses, case studies or more 
detailed explorations of ‘learning events’ or local experience to provide rich data from 
which to make decisions (Whitehurst, 2004). Small scale, well-organised in-depth 
studies can illustrate good practice and provide useful insights into teaching and 
learning. In this review we have included case studies and mixed method studies that 
are not experimental or correlational, but which have significant research rigour and 
where the results are congruent with what we know about adult learning and literacy 
and language learning. 

Of course, much of what teachers do has not been researched, nor is it ever likely to 
be. Teachers will continue to use experience, and local knowledge to develop, adapt 
and enhance teaching practices and programmes to benefit learners. However, 
taking quality research findings into account in planning and delivering LNL teaching 
is likely to enhance the opportunities learners have to make real progress. 



1.4 Reading research in the light of the local context 

When considering the applicability of international literature and research findings to 
New Zealand, the particular characteristics of learners and provision in this country 
need to be taken into account. 



1.4.1 The nature of adult LNL learners in New Zealand 

The nature of the learners themselves is one of the key features of teaching adult 
literacy, numeracy and language. Many of the characteristics of learners portrayed in 
the literature reviewed during this study appear to be similar to those for learners in 
New Zealand. 



8 An effect size is the percentage of the standard deviation from the pre-test. Effect sizes are relative 
measures of how much improvement over a control group an intervention yields (Kamil, 2004). 
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It is highly probable (although not been validated through specific local research) that 
LNL learners here will have ‘spiky’ skills profiles similar to those overseas (Besser et 
al., 2004; Grief & Windsor, 2002; Strucker & Davidson, 2003), which means they will 
have quite diverse needs. Many English-speaking learners have much better reading 
than writing skills; some have better oral/aural skills than others. Some ESOL 
learners speak English, but have never learned to read or write the language; others 
have no literacy skills in any language and very beginning spoken English skills. 

The confidence learners bring to the task of learning LNL also varies considerably. 
Some adults who need additional LNL skills because of changes in employment may 
be confident about themselves as learners. Others may have been unsuccessful at 
school and have extensive feelings of negativity if not outright antagonism, towards 
schools and teachers (Benseman & Tobias, 2003). Adults re-entering the education 
system who perceive themselves as having been unsuccessful at school may be 
apprehensive of receiving ‘more of the same’. Even when their school experiences 
were positive, many of those with low literacy did not participate in post-school 
education; the time lag since studying means some have little confidence in their 
abilities to learn new skills or improve existing ones in a formal educational setting. 
For many people, achieving some degree of self-efficacy 9 is essential to maintaining 
a level of motivation to stay engaged in their learning. These characteristics are 
common to LNL learners throughout the western world. 

ESOL learners are found in programmes specifically catering to them (e.g. in 
dedicated Training Opportunity and Youth Training (TO/YT) courses or community 
programmes), but also in larger numbers in mainstream LNL provision. ESOL 
learners with high levels of literacy in their first language have different learning 
needs from those with very limited language and literacy and it may be more effective 
to teach them separately. Conversely it may not be effective to place ESOL learners 
with low literacy and little language together with adult literacy learners with fluent 
spoken English (Manwaring, 2001 ; Shameem et al., 2002; Suda, 2002a). 

A proportion of learners may have learning disabilities. Over half of those assessed 
in IALS in New Zealand as Level 1 (for prose, document and quantitative domains) 
and over a quarter of those assessed as Level 2 had answered ‘yes’ to the 
supplementary question, “Did you ever have a learning disability?” (Chapman, 
Tunmer, & Allen, 2003). To make the diversity of learners even greater, some LNL 
tutors 10 report anecdotally that the proportions of students with learning issues 
exacerbated by drug and alcohol abuse are increasing, although we have no actual 
data on this. 

On top of these issues as learners, the realities of being parents, workers and family 
members often make sustained periods of learning difficult. The stress of multiple 
roles is an issue for many adult students, but in comparison with other tertiary 
students, LNL students are more likely to be on benefits or have low incomes with 
the resultant strain on resources that study imposes. Issues such as transport, 
childcare, and family health are also major issues for many LNL learners, 
exacerbated by low incomes (Benseman, 2004). ESOL learners may be coming to 



9 Self-efficacy is a belief in one’s ability to carry out the actions necessary to manage particular 
situations. It is more specific and contextualised to learning than self-confidence or self-esteem. For 
further information see http://www.emorv.edu/EDUCATION/mfp/efficacv.html 

10 In this report, the term ‘tutor’ refers to an adult LNL teacher, regardless of whether they are paid or 
voluntary. 
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terms with resettlement issues, which affect their motivation to learn, and some may 
have had no experience of schooling at all. 

School students in New Zealand receive on average 15,000 hours of schooling 
(Hattie, 2002a). It is a real challenge for LNL teachers to help learners acquire or 
enhance their literacy, numeracy and language skills in a programme of a few 
hundred hours (and often much less), when those learners have highly diverse 
motivations and skills profiles, and when many have significant issues about their 
capacity to be effective learners or are learning a new language. 

What is not yet known is what features might distinguish LNL learners in New 
Zealand from learners overseas, and how that might impact on teaching and learning 
processes and the implementation of research-based interventions. 



1.4.2 The nature of literacy, numeracy and language provision in 
New Zealand 

A second feature of LNL provision is the diversity of providers and contexts in which 
provision occurs (Cain Johnson & Benseman, in press). Historically provision was 
dominated by only a few organisations (i.e. Literacy Aotearoa - formerly the Adult 
Reading and Learning Assistance Federation (ARLA), Workbase: The New Zealand 
Centre for Workforce Literacy Development, and the National Association of ESOL 
Home Tutor Schemes). Now, a diverse range of organisations and institutions 
provide LNL, including Private Training Establishments (PTEs), private educational 
companies, Tertiary Education Institutions (TEIs) such as polytechnics and wananga 
and increasingly, adult and community education providers (the ACE sector). Some 
of these providers have LNL as core business; others provide it as an adjunct to 
other teaching. 

Provision currently occurs in a wide range of contexts: the community, workplaces, 
prisons, unemployed programmes, private homes, community organisations and 
churches and is funded out of a correspondingly diverse range of sources 
(predominantly government departments). Teaching takes place under a range of 
titles including: adult literacy, numeracy, ESOL, family literacy, whanau literacy, 
foundation skills, learning support and bridging education. 

The learners are taught using a range of methods (about which we still know very 
little) and can receive anything from less than one hour to 30 hours of tuition a week. 
The actual extent of ‘direct acts of teaching’ in any of these programmes is not yet 
known. Learners can be taught on a one-to-one basis, in small groups or classes of 
up to 20 learners. Assessment can range from informal verbal encouragement to 
formal tasks measured against the performance criteria for unit standards. 

In general terms, LNL provision in New Zealand has tended to draw more on the 
British tradition of learner-centred curricula and many programmes do not use 
textbooks or workbooks. In contrast, the American model is heavily influenced by 
GED 11 preparation and programmes “often rely extensively on reading texts and 



11 The General Educational Development (GED) is a test in maths, English, social studies and taken by 
individuals who do not have a high school diploma. Passing the test implies that the taker holds the 
same knowledge he/she might hold having completed high school in the US. Much of the foundation 
learning-related education in the US is geared toward learners passing their GED. 
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workbooks cloned from elementary schools” (Gretes & Green, p. 28). The curriculum 
for ESOL provision in the USA has focused more on the challenges immigrants and 
refugees face in negotiating English in every day life but still with extensive use of 
textbooks. 

We have very little systematic knowledge about those who teach LNL to those 
diverse learners in the plethora of contexts and providers discussed above and how 
their skills, knowledge and attributes compare with those undertaking similar teaching 
overseas. Adult literacy tutors historically have been predominantly Pakeha women 
volunteers (Hill, 1990); the workforce may now somewhat more diverse, but as yet 
there is no comprehensive picture. In one of only a few studies that has collected 
data on tutors (Sutton, 2004) a sample of 80 tutors showed 85% were women, 75% 
Pakeha, 65% were over 40 and approximately 42% had full-time LNL-related jobs. 

There is no standardised training system for LNL teachers/tutors and many teachers 
have minimal levels of formal LNL-related training relative to other educational 
sectors. In Sutton’s study for example, only 17% were categorised as having 
teaching and specialist adult literacy qualifications. 

Important aspects of programme organisation may differ between programmes here 
and those overseas, which might impact on how research is undertaken and 
implemented e.g. hours of work (full/part time employment) contact time versus 
preparation time, access to ICT, professional development opportunities etc. 

Our professional experience suggests there may be other (but as yet unverified) 
areas of difference between New Zealand and overseas including: 

> higher retention in some New Zealand programmes than overseas; however, 
we have little data on this and none on the regularity of attendance of learners 

> leaving for employment is considered as a positive outcome for vocational 
programmes here, regardless of the extent of any LNL skills gain 

> the opportunity for more learner-centred curricula here (certainly in some 
community-based provision) than seems to be evident in the overseas 
literature and in a related vein the limited use of textbooks 

> an emphasis here on culturally appropriate provision (for Maori and Pasifika 
learners); we did not identify any studies meeting the criteria for this review 
that related to programmes for indigenous people 

> New Zealand’s historical pattern of voluntary participation, compared with the 
degree of compulsion apparent in the USA where continued government 
assistance may be linked to participation in literacy programme, for example 
for parents receiving welfare or ex-inmates or young offenders who are 
signed up to literacy programmes as a condition of their not going (back) to 
prison 

> less differentiation of provision based on formalised assessments and 
learners’ literacy or language levels (compared with ABE /GED distinctions in 
the USA and National Standards in the UK) 

> in New Zealand vocational and workplace literacy provision is very evident in 
the sector alongside provision in tertiary institutions, compared with for 
example the USA where vocational provision is far less visible within the 
ABE/ESOL sector 
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> tutors who have less formal LNL training and fewer professional development 
opportunities than teachers working in some programmes overseas 

As our research programme unfolds over the next few years we will gain greater 
insight into our distinctiveness. 
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2 PROJECT METHODOLOGY 



Every evidence-based review adopts a different methodology, depending on the 
nature of the sector involved and the literature that is identified during the review 
process. As the process outlined below will illustrate, our initial methodology has 
evolved to take into account the current characteristics of LNL research, in keeping 
with Alton-Lee’s (2003) view that developing a methodology for providing a research 
synthesis is an iterative process. 

This is the first study in New Zealand that looks systematically at adult literacy, 
numeracy and language teaching and learning and its purpose is to inform policy 
development in a relatively new part of tertiary education. 12 Therefore, we have 
included more explanatory information about the components of literacy, numeracy 
and language teaching, more background information about particular research 
studies and have been more inclusive of studies than might otherwise have been the 
case, in order to provide readers with a fuller understanding of the field. 



2.1 Literature retrieval 

The first stage of searching was carried out by two experienced database 
researchers/librarians, using the initial set of criteria as stipulated in the research 
contract which were: 

> Research studies in English where outcomes were clearly linked to 
interventions or programme characteristics. ‘Outcomes’ included: 

• changes in literacy, numeracy and language 13 skills (both tested and 
self-reported, but with a preference for the former) 

• gains in employment, acquisition of a better job, increased income, 
participation in other education, reduced reliance on welfare, 
achievement of educational qualifications, increased involvement in 
children’s education 

• changes in self-confidence, self-esteem or self-efficacy 

• achievement of learners’ personal goals 

• any other outcomes that may need to be considered in this review, 
and in any future programmes. 



> Studies of sufficient quality to enable statements of confidence about their 
generalisability beyond specific contexts (including refereed journals, doctoral 
or masters theses). 



12 LNL teaching is a relative newcomer to the educational world, both in New Zealand (Hill, 1990) and 
internationally (Hamilton & Merrifield, 2000; Kett, 1994). 

13 The scope of our study did not include Second Language Acquisition (SLA), the study of how people 
acquire a language other than their mother tongue. SLA investigates ‘such factors as age, aptitude, 
personality, motivation, cognitive style and learning strategies’ (Barton & Pitt, 2003, p. 9). SLA is more 
often aligned to linguistics than education. Readers interested in how people acquire a second language 
could refer to Second Language Acquisition (Ellis, 1997). 
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> Studies published since 1990, unless regarded as landmark studies. 



We were looking for information about a broad range of factors that may have 
influenced learner outcomes, including: 

> Tutor/educator factors: 

• tutor/educator educational experience 

• pedagogical characteristics and theoretical approach to teaching and 
learning (with reference to different learning environments) 

• curriculum planning and preparation 

• background and identity characteristics (e.g. gender, ethnicity). 

> Learning environment factors that directly influence the teaching and 
learning process including: 

• curriculum content 

• assessment practices (including initial diagnostic assessment) 

• curriculum delivery environment (including combinations of teaching 
teams, use of pedagogical resources, class size and student-teacher 
ratio, combinations of contexts) 

• optimum intensity of learning and duration of engagement in learning. 



Any evidence regarding the characteristics of effective policies in respect of the 
identified factors influencing outcomes was also sought. 

Appendix B provides a complete list of the databases and web-sites that were 
covered in this search; Appendix C lists the keyword descriptors used in the 
searches. Where the descriptors did not match for a particular database, comparable 
synonyms were used. 

The search identified studies primarily from the USA and Britain. We then looked 
more specifically to the literature from Australia and Canada but found very little 
research that met our criteria. 

A total in excess of 300 articles and reports were identified at this first stage and 
entered on an EndNote database for initial consideration. 



2.2 Reviewing initial selection 

In the second stage, at least two of the research team (Benseman, Sutton and 
Lander) independently read the abstracts and/or full texts of the identified articles and 
reports and evaluated them using the criteria listed above, plus additional criteria 
appropriate to the type of study concerned. This second stage of the search 
eliminated more than 200 of the initial list, usually because of a combination of 
factors: 

> evaluations not related to any specific element of the programme 
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> inadequate reporting of outcomes 

> multiple interventions used, thereby precluding the ability to differentiate 
between the interventions 

> intervention not specified, unclear or unable to be replicated 

> very small sample sizes 

> very limited information about methodology or results 

> poor data collection, usually due to very high rates of attrition 14 

> poor linking to outcomes 

> inaccessibility of the full report/article 

> ICT 15 interventions likely to be technologically dated. 

Because of the small number of studies located (relative to most literature searches), 
the researchers erred on the side of inclusion at this stage to ensure even a 
moderate number of studies for consideration. All of the studies meeting the criteria 
were then requested through Interloan or downloaded in full off Web sources. 



2.3 Scrutiny of selected studies 

The process above identified approximately 70 individual studies suitable for close 
investigation. At this point, the NRDC posted on their web-site a research review of 
randomised controlled trials (RCTs) and controlled trials (CTs) in adult literacy and 
numeracy (Torgerson et al., 2004). This review initially identified 4,555 studies using 
very broad descriptors; of those 149 studies were of sufficient quality to be read in full 
and yet only 36 RCTs and CTs had sufficient high quality data to be reviewed by 
experts. Of the 36, only six showed clearly attributable positive outcomes. 

We had located the majority of those studies already ourselves in our search and 
initially thought that because of the rigorous process used for critiquing the studies 
that were included, we would be able to use the NRDC analysis as a shortcut for part 
of our own review. 

However, on closer reading, the Torgerson review had several significant limitations 
that meant we had to look more broadly. Firstly, it only covered randomised 
controlled trials (RCTs) and controlled trials (CTs), excluding all other evaluations, 
case studies and other correlational studies, which meant the body of material on 
which we could draw was very small. Secondly, as already discussed in Section 1 .3, 
some of the studies were of dubious quality. Thirdly, some of the studies were clearly 
dated (e.g. computer-based programmes from the early 1980s). Fourthly, the findings 
were not presented in a useful format for policy-makers or practitioners, particularly in 
terms of the implications arising from the findings. Their study did not include English 
for speakers of other languages (ESOL). Finally, some of the contexts were not 
applicable to the New Zealand foundation learning sector. 



14 Adult literacy programmes often have high withdrawal rates, making it difficult to achieve high 
completion rates in studies (B. A. Quigley & Uhland, 2000). 

15 The use of computer technology and software to teach literacy 
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A major draft literature review of teaching interventions for adult ESOL learners 
(Condelli, 2003; Condelli & Wrigley, 2004) was then located. This review identified 17 
key studies in adult basic education (nine of which are included in the NRDC study). 
All 17 studies were reviewed against 43 criteria and the author’s observations about 
the quality of those studies also called into question the merits of taking a narrow 
evidence-based approach: 

The 17 studies identified reflect a haphazard and unorganized approach 
toward studying adult literacy and are not guided by any theory, approach or 
school of thought about good pedagogy, (p. 9) 

The authors then took a more traditional approach to reviewing studies relevant to 
adult ESOL literacy, drawing on second language acquisition and ESOL literature. 
This review informed our discussion in the ESOL section of our report. 

To enhance the usefulness of this review and in keeping with the broader concept of 
‘evidenced-based’ research we had adopted, we then sought other sorts of research, 
which were screened according to a number of factors: 

> whether there was sufficient information about methodology to make it 
possible to replicate the study 

> whether the research appeared to be congruent with New Zealand contexts 

> publication in referred journals 

> favouring multi-site/multi-agency studies, over single institution research 

> the reputation of the organisation or researcher conducting the study 

> whether the methodology was transparent and appropriately linked to the 
findings. 



If we had maintained our original inclusion criteria, we would have had a very limited 
number of studies to review and even those would not have been of sufficient quality 
to satisfy the criteria for most evidence-based literature reviews. In a small number of 
cases we have included reports that are based on the ‘professional wisdom’ of 
groups of practitioners, when little else is known. 

While in some instances the research base is thin, large-scale research studies 
currently underway are building a much richer and more detailed picture of adult LNL 
provision around the world. Findings from these studies, although not based on 
samples on the scale available in the schooling sector, 16 do point to teaching and 
learning practices that are effective and enhance learners’ opportunities to make 
gain. 



16 A recent synthesis of studies on effective schooling reviewed over 500,000 studies, involving results 
from millions of learners (Hattie, 2002b). 
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2.4 Assessment instruments used to determine change 

The validity of the assessment instruments used to monitor changes in participants’ 
LNL skills is central to the identification of factors influencing LNL outcomes. Without 
valid, reliable measures of outcomes, the ability to demonstrate effectiveness of any 
educational element is severely limited. 

The design and use of assessment instruments are contentious not only in New 
Zealand, but also internationally. There is considerable opposition from practitioners 
and learners to the notion of assessment testing, predominantly because of the 
perceived lack of appropriate tests and a negative association with schooling - a 
spectre that most practitioners strive to overcome with their learners. In the US (often 
seen as the ‘home of tests and testing’), there is considerable use of LNL 
assessment tests, but this widespread practice occurs predominantly because of 
funding requirements, rather than any true acceptance of their validity or usefulness. 
Indeed, most practitioners use the tests under duress rather than out of professional 
choice (Ehringhaus, 1991). 

In New Zealand, there are no standardised adult-specific LNL assessment tools 
available and researchers have been known in the past to resort to tests such as the 
Burt Word Reading Test, which was designed and standardised for school children. 

Assessment can be done for a number of purposes: 

> to initially identify learners who need help (screening) 

> to analyse specific difficulties in order to design a learning programme 
appropriate to the needs of the individual learner (diagnostic) 

> for recording learner progress over a given period or on exit from a 
programme 

> to map learner skill levels against national or international standards 

> to certify certain levels of competence have been reached (i.e. as in the US 
initiative Equipped for the Future which offers a ‘work readiness credential’) 

> various combinations of the above. 

Assessment tools for the first three purposes (screening, diagnostic and learner 
progress) are often tailor-made by teachers or institutions, whereas the latter 
purposes are served by professionally-designed, standardised tests. The most 
commonly used of these standardised tests in the US are the Test of Adult Basic 
Education (TABE), the Comprehensive Adult Student Assessment System (CASAS) 
and the Test of Applied Literacy Skills ( TALS ), which are most commonly used for 
funder reporting purposes, but can be used by teachers for their individual learners. 17 
A number of the studies reviewed mentioned a problem of resistance to testing (see 
for example, Batchelder & Rachal, 2000) and despite their widespread usage, these 
tests are frequently criticised for their inability to measure the ‘true achievements’ of 
programmes (Sticht, 1999). 

Beder (1999) has made the point, that despite all the research undertaken to date, 
the field is still left with a contradiction in regard to determining whether learners have 
made gains. Learners themselves perceive they gain in skills, but the test results 



17 For a comprehensive discussion of assessment see Kruidenier (2002a). 
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used to measure improvement do not necessarily reflect these gains. Do learners 
inflate their self-report because it is socially desirable to gain literacy, or are they 
making important gains that tests are not sensitive enough to measure? If learners 
are right, perhaps major gains are contextual and personal. If the tests are right, the 
quality of teaching and staff development needs to be examined. Either way more 
quality research is needed about how, or what, gain is measured. 

In Britain, a recent review of standardised assessment instruments (Torgerson et al., 
2004) concluded that none of the 15 instruments that have been used in that country 
over the past decade met the quality criteria required to make them appropriate for 
use in research. The National Foundation for Educational Research in the UK has 
therefore been commissioned to develop a new instrument suitable for adult literacy 
assessment in research projects and there are also plans to develop one for adult 
numeracy. 

A survey of the assessment instruments and processes used by New Zealand 
practitioners is due to be published during 2005 and should provide useful 
information about current patterns and associated issues. 
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3 RESEARCH FINDINGS 



This section describes what has been ascertained from the literature about the 
factors that appear to be important to ensure quality learner outcomes, organised in 
five themes: quality; participation and retention; characteristics of provision (e.g. 
curriculum, whether provision is individualised or in groups, the impact of ICT); the 
specific components of LNL (reading, writing, ESOL, and numeracy); and a 
discussion of the different contexts in which LNL provision takes place. 



3.1 QUALITY 

The research on quality fell into three groupings: what sort of programmes were 
effective, what factors were considered necessary for quality programmes and 
learners’ perspectives on these topics. 



3.1.1 Effectiveness of programmes 

While it is almost universally assumed that LNL interventions are more effective than 
not providing any interventions at all 18 , it is perhaps useful to start by addressing this 
fundamental question before addressing how the impacts were achieved. 

The literature does point to programmes having an impact, although there is not a 
great deal of research on this topic. A NCSALL study (Beder, 1999) identified adult 
literacy research conducted in the USA since the late 1960s that identified outcomes 
and impacts for learners. While the report is not particularly helpful for the central 
purposes of our review (because it is not evaluating organisational practices or 
pedagogical interventions that lead to those outcomes), it does point to programmes 
making a difference. From 115 studies, 23 were considered credible enough to 
analyse and from which to make tentative conclusions about the effectiveness of the 
adult literacy education program in the United States: 

> In general, it is likely that participants in adult literacy education receive gains 
in employment. 

> In general, participants in adult literacy education believe their jobs improve 
over time. However, there is insufficient evidence to conclude that 
participation in adult literacy education causes job improvement. 

> In general, it is likely that participation in adult literacy education results in 
earnings gain. 

> In general, adult literacy education has a positive influence on participants’ 
continued education. 



18 In their review of RCTs and CTs, Torgerson et al. (2004) point out that having control groups that 
receive no teaching (as opposed to conventional teaching) is pointless, as an intervention/no 
intervention at all comparison is rarely a policy option; rather, the options are usually between different 
types of intervention. 
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> Although the evidence suggests that participants in welfare-sponsored (e.g. 
JOBS Program) adult literacy education do experience a reduction in welfare 
dependence, the evidence is inconclusive as to whether adult literacy 
education in general reduces welfare dependence for participants. 

> Learners perceive that participation in adult literacy education improves their 
skills in reading, writing, and mathematics. 

> As measured by tests, the evidence is insufficient to determine whether 
participants in adult literacy education gain in basic skills. 

> In general, adult literacy education provides gains in GED acquisition for 
participants entering at the adult secondary (ASE) level. 

> Participation in adult literacy has a positive impact on learners’ self-image. 

> According to learners’ self-reports, participation in adult literacy education has 
a positive impact on parents’ involvement in their children’s education. 

> Learners perceive that their personal goals are achieved through participation 
in adult literacy education. 



More recently, Brooks et al.’s review of British research of programme effectiveness 
(2001, p. 124) concluded that when lessons from both the UK and the USA are 
combined: 

> There was plausible evidence that basic skills tuition benefited students’ 
reading and writing. 

> The average gains in reading and writing in general basic skills provision 
were undramatic but worthwhile. 

> The evidence on the impact of general adult numeracy tuition was sparse and 
unreliable. 

> There was a small amount of attainment evidence showing impact on ESOL 
and ICT. 



3.1.2 Quality of programmes and teachers 

The Basic Skills Agency (BSA) report Effective Basic Skill Provision for Adults (Basic 
Skills Agency, 2000) is based on BSA action research evaluations carried out 
between 1975 and 1997, evidence from BSA’s consultancy experience from 1985 to 
1998 and research on effectiveness funded by the agency. 19 One part of this report 
summarises what they believe to be the hallmarks of quality programmes: 

> Programmes that deliver clearly structured teaching in literacy and numeracy 
(using a mixture of teaching methods). 

> Programmes that deliver skills acquisition in a range of contexts that meet the 
motivation and interests of learners specifically related to that context. 



19 Although these studies are not specifically identified, the BSA has earned a reputation in Britain as a 
credible, high quality agency in the adult literacy field and therefore the analysis has credibility. 
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> Programmes that have high expectations of learners’ achievements have 
higher levels of attendance, completion and outcomes. 

> Programmes that produce for each learner a learning plan that lists and 
provides activities and material to meet specified individual need (including 
learner aims, learner perceptions of strengths and weaknesses, priority areas 
for development, skills broken down into manageable steps with appropriate 
learning strategies, an estimated timeframe, the resources needed and a way 
of reviewing and recording progress). 

> Programmes that regularly assess and review learner progress and adjust 
individual learning plans accordingly. 

> Programmes that enable learners to gain credit and accreditation for their 
learning and enable progression. 

> Programmes that adjust the length of programme according to the level of 
skills required, including a realistic amount of tuition time (p. II). 20 



The BSA report also lists ways in which effective programmes make use of staff: 

> Adequate time for staff to plan and co-ordinate provision. 

> Limited reliance on teachers working 2-4 hours week. Full-time teaching staff 
ensure better consistency of teaching methods and better use of professional 
development resources. 

> Teachers having a recognised qualification appropriate to the field. 

> Teachers need to have credibility in the context in which they are teaching 
(particularly important for vocational programmes where teachers need a set 
of job-related competencies and skills in addition to LNL teaching). 

> Using volunteers to support learners providing they undergo a rigorous 
selection process, receive both initial and in-service training, are supervised 
and supported by paid staff and make a minimum commitment to the 
programme. 



Another British study investigated the LNL progress learners made over two years 
(G. Brooks et al., 2001). The sample included 2,135 learners and 177 tutors with a 
57% retention of learners between pre- and post-testing. 21 The learners were drawn 
from ‘normal’, on-going basic skills programmes. Most received 30 hours of tuition 
between pre- and post-testing, had reading levels similar to nine-year-olds, poor 
qualifications and were judged by their tutors to range from ‘very weak’ to ‘about the 
national average’. Overall, the students (including all sub-groups) achieved a small, 
but statistically significant improvement in reading and writing. The progress was 
judged to be “undramatic, but worthwhile” (p. 1). The study concluded that two of the 
three factors associated with learners making progress were related to the teachers: 
the tutors having qualified teacher status and where they had additional help 
(volunteers or paid assistants); the third factor was the amount of tuition - see 
Section 4.3 below. 



Z0 Many of these appear generic and quality general adult education programmes would display similar 
characteristics. 

21 This study has been strongly criticised for its attrition rate, use of inappropriate tests, inclusion of 
inappropriate learners (skill levels too high) and drawing incorrect conclusions (Hamilton, 2001). Even 
so, it remains the only national study of its type and is worth noting. 
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A New Zealand study (Benseman, 2001) of 13 Training Opportunities tutors who 
were rated as effective teachers included some tutors who were teaching literacy and 
numeracy, although they were primarily teaching other content areas. Although not 
on literacy teachers per se, this study is included because of its New Zealand context 
and the similarities between the learners involved and typical LNL learners. The 
study identified the following characteristics and teaching practices of these 
successful teachers: 

> their overwhelmingly positive attitudes towards their learners 

> accentuating learners’ strengths 

> their use of the learners’ experiences in teaching 

> their ability to create a positive and supportive learning environment 

> their awareness and management of critical periods in the programme (when 
learners were susceptible to failure and/or withdrawal) 

> their passion for their subject, expertise in their subject and links to industry 22 

> being ‘open’ as people to their students 

> providing pastoral care in times of need and crisis 

> teaching learners how to set realistic long-term goals 

> their ability to balance challenge and support for learners 

> using specific teaching strategies and methods they had developed and 
refined. 

In New Zealand, the TEC has developed its own Indicators of Good Practice in 
Literacy Provision in Training Opportunities and Youth Training (Tertiary Education 
Commission, 2003a). These guidelines (for general vocational programmes) were 
written by a small group of literacy practitioners and programme administrators based 
on professional wisdom rather than empirical evidence. The principal actions 
providers are expected to undertake include: 

> having a literacy policy/vision/mission statement based on a definition of 
literacy 

> including literacy in their quality management system 

> providing a ‘literacy friendly’ environment 

> ensuring there are literacy ‘drivers’ (individuals who take responsibility for 
development of literacy in the organisation) 

> planning for professional development 

> enabling staff to access professional and support networks 

> ensuring tutors have access to resources 

> providing tutor non-contact time allocation (for planning and preparation) 

> understanding the literacy demands of programmes and providing learning 
opportunities to meet them 

> undertaking robust initial assessment and learning plans 



Relevant to vocational rather than LNL specialist tutors. 
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> recording literacy skill gain 

> ensuring tutors have learner-centred teaching strategies 

> assistance for learners with high literacy needs. 

Apart from specifying that learning strategies need to be ‘learner-centred’, there are 
no specific teaching methods or strategies identified in the list and overall, less 
emphasis on programmes providing specific teaching and learning opportunities than 
those in the UK, which refer to literacy-focused programmes. 

The findings of the studies above emphasise the importance of effective initial 
training for tutors 23 and comprehensive professional development programmes. 24 

It is worth noting that the retention of qualified, skilled LNL teachers is often a major 
issue for LNL providers. With poor pay, employment conditions and minimal career 
prospects, skilled practitioners are often ‘poached’ by providers in more mainstream 
forms of provision. This situation is especially true of community-based programmes 
(May, Hill, & Donaghy, 2004). 

It is not easy to meet the complex needs of learners but not all has to do with the 
quality of teaching. A policy-oriented and descriptively rich case study of three 
learners (Venezky, Sabatini, Brooks, & Carino, 1996) illustrated how difficult it was 
for learners to make progress. 25 The teachers were confronted with a wider range of 
abilities in their classrooms than they could teach effectively and had little, if any, 
diagnostic information to guide their individual instruction, in a context made more 
complex by open entry/open exit policies and a desire to have separate learning 
plans and materials for each student. 

Their observations revealed a number of less than optimum practices that contrast to 
the elements of effective practice outlined earlier, many of which relate to 
organisation and resourcing. These elements included diagnostic assessment not 
integrated into curricula and teaching programmes; students were not given any 
opportunity to develop fluency; more students than tutors could handle. Most 
students had some aspect of learning difficulty that required individual assistance 
that they were unlikely to receive; students needed practice in holistic tasks, not just 
the component parts contained in pre-purchased material; and too much concern 
was paid to self-esteem to the extent that students were not challenged to attempt 
things that were new, uncomfortable but also necessary for their skills development. 



23 Several publications would be useful resources for the creation of training programmes in New 
Zealand. The first is a recent NRDC study of recent initial teacher training programmes for LNL (Lucas 
2004); the others relate to a NCSALL study of 100 ABE teachers as they take part in a variety of 
professional development strategies (C. Smith & Hofer, 2003). 

24 In the recent trial of the draft Adult Literacy Achievement Framework (Sutton, 2004) tutors self- 
identified the need for more training in literacy teaching. 

25 Thirteen students were studied in depth but only three written about in this report. Although a very 
small sample, it has been included because it serves as a model for rich case studies, supplying insights 
into learners, the programme and organisational matters that are impacted on by programme and 
funding policies. 
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3.1.3 Learners' perspectives on effectiveness 

Few literature reviews of compulsory education incorporate learner perspectives on 
the factors they perceive as relating to effectiveness, because in schools with 
compulsory attendance requirements, learner perspectives are usually peripheral to 
the debate. In LNL, where voluntary participation is usually the norm (at least in New 
Zealand) we believe learner perspectives warrant greater consideration. If adult 
learners do not rate the programme positively, there is an increased likelihood of 
withdrawal or erratic attendance - both of which clearly limit the likelihood of positive 
learning outcomes. 

It must be acknowledged that acquiring quality data on learner perspectives is 
problematic, especially in terms of validity. For example, (as we mentioned in section 
2.2) learner self-assessments of literacy skills often vary from standardised test 
results (the former usually under-rate difficulties); these discrepancies can be seen 
for example in the IALS findings where less than a third of those in the lowest level 
self-rated their skills as either ‘poor’ or even ‘moderate’ (OECD, 1997). While some 
researchers (Sticht, 1999) still believe that learner self-assessments are a valid 
process, others are less convinced (Jones, 1997). When investigating retention and 
drop-out (Quigley & Uhland, 2000) where researchers have probed beyond initial 
reasons for leaving a programme it has often emerged that the learners simply did 
not like the teaching style of the literacy instructor and felt that it was ineffective. 
ESOL learners’ voices are also often missing, largely due to the difficulty and cost of 
interviewing and translating languages other than English. 

Research incorporating learner perspectives tends to fall into two main categories: 
what specific strategies and factors learners report as helping their learning; and 
studies that explore learners’ perspectives on what types of impact LNL participation 
has on their lives and skills. We have endeavoured to focus on the former in this 
review, but the latter is important in that these sorts of studies (e.g. (Bingman, Ebert, 
& Bell, 2002; Bingman, Ebert, & Smith, 1999) provide insights into what learners take 
away from programmes, irrespective of what funders and teachers intend. The most 
obvious example is the consistent outcome of increased self-confidence and self- 
efficacy, often independent of any gains in LNL skills (Benseman & Tobias, 2003), 
which leads some commentators to challenge whether LNL are more about changing 
learners’ life circumstances than teaching LNL skills perse. 

A small number of studies explicitly asked learners what aspects of programmes they 
considered effective. Following an evaluative study of an Even Start Family Literacy 
programme (Yaffe & Williams, 1998), the researchers returned to interview six female 
participants’ to identify what they thought made the programme successful. The 
women said that “the most attractive feature” was the all-female environment, 26 
which they found “trusting and supportive” (p. 13). A close second was the removal of 
barriers to attendance such as “the convenience of having free child-care at the 
same site as the adult education programme” (p. 14). The factors associated with 
learning mentioned by all or most of the women included “individualized instruction 
and assistance in studying” (p. 15). 

A case study of 30 learners and ten tutors from a basic skills centre in the UK 
(Eldred, 2002) sought to discover what they thought about successful teaching and 



26 With such a small sample, the authors restricted their discussion to this particular Even Start 
programme and indicated that further research is required to determine if this “theme has implications 
for other adult literacy programmes”(p. 17). 
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learning. 27 For the second part of the study, when asked what leads to success, the 
majority of learner responses were about "the skills of the tutor... and the fact that 
tuition was not like school” (p. 16). They talked about ‘‘the relationship with the tutor, 
the learning environment and methods” (p. 23). Unfortunately, little space is given to 
presenting or discussing these findings in the report. The finding that only 10% of 
learners identified regular attendance as a being a factor that helped them 
succeed 28 , suggests that research into what is not important to learners may be 
equally useful to programme planners and tutors. 

More recently, a British study (Ward, 2003) specifically set out to record learners’ 
perspectives on achievement and progress. Researchers interviewed seventy adults, 
using individual and group semi-structured interviews and a ‘learning journey’ 
metaphor to facilitate these discussions. Whilst there are no comments relating to 
effective teaching and learning strategies, the results are a rich source of 
programme-related factors that learners identify as contributing to successful 
learning. One set of factors is linked to ‘starting out’: those with the most involvement 
in planning tended to be the most satisfied with their progress (p. 27). Another set of 
factors relates to the context: in a class of 40 learners with two or three tutors, the 
learners felt they were achieving very little because of the lack of individual tuition. 
Peer support in the form of sharing skills and ideas, and friendships, were important 
to learning and achievement. Progress was inhibited where learners were 
uncomfortable with the constant flow of people around them in a busy Centre (p. 31). 

The relationship between tutors and learners was a crucial element of 
learning and most respondents valued and respected their tutors and 
trusted their expertise. Tutors’ attitudes were really significant, as learners 
wanted a tutor who made them feel as though they were in charge of their 
own learning.... Learners felt it was important to feel on the same level as 
their tutor or they wouldn’t learn, and progress was viewed as more shaky 
where negative relationships were reported (p. 37). 

Praise of “the effort devoted to gaining achievement was a significant factor in 
learning, particularly as a motivational tool.” Encouragement was noted as “a key 
factor in building confidence and self-esteem... Conversely, lack of interest in their 
achievements or dismissive remarks were profoundly discouraging” (p. 38). 

Finally, the learners linked assessment practices with effectiveness. They reported 
that self-assessment and the tutor’s judgement of their achievements (most 
preferring constructive, verbal feedback) were supportive of their learning. Most 
learners preferred assessment to be “an ongoing process based on discussion and 
portfolio building supported by tutor feedback and individual reflection, particularly 
where skills acquisition and the ability to do things in their lives were the main aims" 
(p. 40). 

Other studies did not explicitly seek learner perspectives on effectiveness, yet 
reported learner comments on teaching and learning strategies that were considered 
valuable. This group of studies includes action research undertaken in the US 



27 Although a very small sample, light on an explanation and justification of their study’s design, it does 
appear ‘fit for purpose’ and is commenting on an area little researched. 

28 A finding that stands in opposition to most other studies on this dimension of effectiveness 
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(Lunsford, 2001). It is of limited applicability 29 yet in a field where learner ‘invisibility’ 
in research is common, a brief mention is warranted. Within the context of a GED 
programme, Lunsford changed from a “teacher-directed, pedagogical approach to a 
facilitative andragogical approach.” When learners were asked what they thought 
about the change, one of the main comments was that they felt motivated by the 
personalized attention they were receiving. 

King and Wright’s (2003) study collected data from 19 ABE learners who volunteered 
to attend interviews. The multiple-choice, open-ended, and exploratory probe 
questions included some comments on factors associated with success as a minor 
part of the study. 30 The researchers briefly comment on one theme - that learners 
identified “learning taking place with and because of community support” (p. 112); 
that is, working together in teams or small groups helped them learn. More 
specifically, learners said that they associated this type of environment with having 
someone to ask for help, helpful discussions, being listened to, sharing experiences, 
and having fun. 

A follow-up study of adult literacy learners in Christchurch (Benseman & Tobias, 
2003) included some observations by the participants about the programmes’ 
strengths from their perspectives as learners. These elements included the ‘non- 
school’ environment where they felt respected, listened to and generally treated as 
adults. Other pluses were the tutors who went out of their way to support the learners 
in times of need or crisis, working in small groups and the tailoring of programmes to 
individual learners’ needs. 



3.1.4 Discussion 

There is evidence that providing tuition for LNL learners is more effective than not, 
although this evidence is both sparse and not available in all areas. 

There is a considerable degree of commonality about the components of what is 
regarded as quality provision from researchers and study results. Key characteristics 
include highly trained staff who relate positively to learners, explicit teaching of 
literacy and programmes that provide individualised tuition. Learners also confirm the 
importance of qualified and supportive staff. The findings that relate to the need for 
appropriately qualified and trained teachers are in line with issues already identified 
in New Zealand. 

There has been a growing emphasis in New Zealand for contextualised teaching and 
individualised learning plans, in a similar vein to the outline of quality provision 
presented in the BSA (2000) report. Nevertheless, the issues raised by Venezky et al 
(1996) about the reality teachers face in the light of these policies need to be borne in 
mind. The diverse and complex nature of learners LNL needs place great demands 
on teachers, impact on the ability of programmes to deliver, and we know nothing 
about the impact of open entry on learner achievement. 



29 The report lacks sufficient description of the method to be assessed or replicated, and only 15 
learners appear to be involved in this study. Also, the reporting of learners' perspectives relied on 
“occasional informal discussions” with the group at the start of the project and then the tutor's journaling 
following one to one meetings rather than direct recording methods which have greater credibility. 

30 The study’s focus was on what facilitates perspective transformation, a particular outcome of adult 
learning proposed initially by Jack Mezirow, whereby learners come to reframe how they understand 
their world. 
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Most of the elements identified in these studies are consistent with what adult 
learning theorists such as Knowles and Brookfield have long advocated: adult 
appropriate environments where learners feel valued and supported, especially in 
times of need. Adults value intimate teaching arrangements where their learning 
needs are explicitly recognised and tuition is tailored to these needs. 

Skilful, supportive teachers who provide on-going feedback are rated highly and 
back-up services such as childcare services help make attendance less difficult. 

Incorporating, or at least being aware of, learners’ perspectives in the design of 
programmes and teaching, is an important component in LNL. As the New Zealand 
LNL research programme develops, ensuring that learners’ perspectives are included 
should be incorporated into methodologies. This perspective needs to be done more 
robustly, than for example, in simple satisfaction surveys. Perspectives from learners 
will be particularly important when considering whether and how provision meets the 
needs of special groups of learners such as Maori, Pasifika, ESOL and young adults. 
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3.2 PARTICIPATION AND RETENTION 

As already discussed, participation is a central issue for LNL because most learners 
attend voluntarily and can vote with their feet. The amount of teaching learners 
actually receive, together with retention are key research issues to review in 
assessing what makes for effective provision. 



3.2.1 Mandatory participation 

While most learners in New Zealand still participate in LNL on a voluntary basis, 
there are increasing indications of a limited degree of compulsion in some 
programmes, for example resulting from court sentences or conditions imposed for 
receiving various government benefits. Compulsion (albeit subtle or involving some 
degree of ‘choice’) is anathema to many practitioners who see the voluntary nature of 
participation as integral to LNL and one of the key features underpinning its 
effectiveness - and in strong contrast to the compulsory schooling system where 
many LNL participants developed their poor levels of self-efficacy. 

An RCT on this aspect of provision (Martinson & Friedlander, 1994) was carried out 
as part of the evaluation of the Californian Greater Avenues for INdependence 
(GAIN). GAIN involved more than 179,000 welfare-to-work participants, of whom two- 
thirds were assessed as needing basic education. These enrolees were then 
required to participate in basic education (ABE, GED and ESOL) or risk temporarily 
losing part of their benefit. The sample for this RCT study contained 581 GAIN 
enrolees and 500 controls from five diverse California counties. 

The study was discussed in Torgeson, et al (2004) but we were unable to source the 
original. The initial results showed no statistically significant change on LNL test 
scores on participants compared to the control group, despite an average 
participation of 500 hours. However, the initial results had included those who had 
enrolled in GAIN, but not actually participated in any education. A re-analysis of the 
results (Boudett & Friedlander, 1997) comparing post test results of participants and 
non-participants found that those with higher pre-test scores (i.e. who had more LNL 
skills to start) benefited more than others. Participation up to 1000 hours of provision 
also appeared to correspond to higher post-test scores. The re-analysis suggested 
that there had to be major improvements in the quality of the programme before any 
significant improvements (particularly in people with low skills) would be seen across 
the (very large) participant population as a whole. 



3.2.2 Amount of participation 

The question ‘how much tuition is enough?’ is akin to asking ‘how long is a piece of 
string?’ yet nonetheless it is an issue of central concern to policy-makers and others. 
Ensuring sufficient time to achieve learning gain to satisfy learners, tutors and 
funders is a basic, but important, quality of effective LNL provision, but is problematic 
in the absence of any agreement on what learners should know or be able to do 
when leaving programmes. This is compounded by the great diversity of skills and 
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knowledge students start with and their different expectations about what level of 
proficiency may be good enough for them. 

Before looking at the various estimates on this issue, it is worth reiterating that many 
LNL learners have already had a decade of schooling, sometimes including some 
additional specialist remedial tutoring. Despite this amount of tuition (or perhaps even 
because of some of it), these learners have not made sufficient progress in their LNL 
skills; changing this situation will not occur easily or quickly in the great majority of 
cases. When the tuition is not intense (e.g. one or two hours a week in many cases- 
40-80 hours total per year) the challenge is even greater for learners and teachers. 

The publication Effective Basic Skills Provision for Adults (Basic Skills Agency, 2000) 
estimates “the number of learning hours (including direct tuition, assessment, 
supervision, guidance and tutorial support) required for learners at different levels of 
attainment to achieve competence in any one level” 31 (p. 19) are: 



Below entry level 


330-450 hours 


Entry level 


210-329 hours 


Level 1 


120-209 hours 



Interpreting this conservatively, a learner would need in excess of 660 hours 
minimum to progress from below entry level through to Level 1 and possibly as much 
as 1000 hours (although the period of time over which this might occur was not 
discussed). 

Brooks et al.’s study quoted above (2001, p. 4) showed that “students who attended 
51-60 hours of literacy tuition between pre- and post-test made the largest gain of 
any subgroup. Since the maximum number of teaching weeks between the tests was 
about 20, these were students who had attended very regularly.” Because students 
on intensive courses attend regularly, the authors tentatively conclude that intensive 
courses “could be particularly effective” in this regard. 

A NCSALL review by Comings (2003) refers to three studies that concluded that 
approximately 100 hours of instruction is needed to achieve an increase of one grade 
level equivalent on a standardised test of reading comprehension. 32 Comings’ own 
research (Comings, Sum & Uvin, 2001) showed that after 150 hours of instruction, 
“the probability of making a one grade level or greater increase was 75%” (p. 10); in 
other words, three of every four learners progressed at least one grade after 150 
hours of tuition, but this amount is not a water-tight guarantee of success as one 
quarter do not make this amount of progress. 

According to Comings, the real issue is that the average student in the US spends 
fewer than 70 hours total in a programme over a 12-month period - less than one 
tenth of what their K-12 school equivalent does. The 70-hour figure is somewhat 
optimistic as adults who drop out before completing 12 hours of instruction are not 
included in this figure. The true average is probably considerable lower. 

A recent major study of ESOL teaching (Condelli, 2003) found that regular 
attendance by ESOL learners appeared to be more important than the amount of 



31 These levels relate to the UK National Standards for adult literacy and numeracy and at this point are 
not comparable to anything in New Zealand. Pre-literate and very beginning readers would be below 
entry level. 

32 This does not equate to levels in the UK standards. 
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tuition per week and that students attending more hours per week gained more in 
comprehension. 33 

There is very limited data on the amount of provision learners receive in New 
Zealand. In one study (Sutton, 2004) more than 50% of the 347 participants in the 
draft ALAF trial received less than 20 hours over the trial period which was a 
maximum of 20 weeks. Extrapolating this data for a year, a learner would receive far 
less than the 100 hours benchmark discussed above. Just under 20% of the sample 
received more than 100 hours tuition in the same time-frame. In the only New 
Zealand study that was identified linking hours to achievement (Shameem et al., 
2002) ESOL learners receiving 12 hours tuition per week (240 hours in total during 
the study) made gains in both reading and writing while those receiving two hours per 
week (40 hours in total) only made progress in reading. 34 



3.2.3 Retention 

Given the basic premise that learners need to attend a reasonable minimum amount 
of tuition in order to make significant gains and that participation is predominantly 
voluntary, 35 the retention of learners becomes a central issue to this debate. Quigley 
and Uhland (2000) claim that “dropout is surely one of the most enigmatic, most 
exasperating, and overall most depressing issues in the entire field of adult literacy” 
Quigley (1997) quotes American data on attrition: funded programmes attract only 
8% of those eligible for them, 20% of those who say they will attend do not show up; 
of those who do, the attrition rate is over 70%. Another study reported by Quigley 
(ibid.) showed a dropout rate of 18% before 12 hours of tuition had been completed, 
20% at 16 weeks and 50% after 16 weeks. A national evaluation in the US (Young & 
et al., 1995) found that most learners leave before completing 100 hours and most 
also report that they have not met their own goals. 

Withdrawal from a programme is not always a negative outcome. A British study of 
1,920 learners (Kambouri & Francis, 1994) showed a 79% annual attendance rate 
overall, although when withdrawals were included, this figure dropped to 40-60%. Of 
those who left, over a third progressed in some way or other to other courses or jobs. 
Another third left for unknown reasons, the remainder cited personal or domestic 
reasons. Rates of actual attendance for persisters and leavers were similar. 

Despite its clear importance, only three studies were identified for our review and the 
topic remains under-researched, especially in terms of specific interventions that help 
increase retention rates. 

A NCSALL study by Comings, Parrella and Soricone (1999) analysed the barriers 
and social and organisational supports that influence whether learners persist in their 
studies. By understanding how positive, supporting influences can be maximised and 
negative, inhibiting factors can be minimised or eliminated, the authors argue that 
practitioners can effectively increase overall retention rates, both at individual and 
programme levels. Based on a literature review, consultations with experienced 
practitioners and interviews with 150 students, the study found that immigrants, 
parents of teenage or grown children and those over the age of 30 were the groups 



33 This study is discussed more fully in Section 3.6.3 

34 See more description of the study in Section 3.6.3 

35 Albeit with some degree of coercion in some cases 
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most likely to persist. Similarly, learners who had been involved previously in basic 
skills education, self-study or workplace training and those who include very specific 
goals were less likely to withdraw. Based on these findings, they identified four key 
supports to persistence: 

> Management of the positive and negative forces that help and hinder 
persistence (especially the strongest ones). 

> Build self-efficacy (belief in one’s ability to learn successfully), not just self- 
confidence) about reaching goals (especially through mastery learning, 
vicarious experiences provided by social models such as former students, 
social persuasion from a culture of support and opportunities to address 
physiological and emotional states). 

> Establishing and revising student goals to use as a context for instruction. 

> Making tangible progress towards the goals. 

The authors make a specific point that policy-makers should hold providers 
accountable for the quality of the strategies and structures that programmes use to 
increase learner retention. 

In a quasi-experimental study of retention, Quigley & Uhland (2000) identified a 
group of 20 ‘at-risk’ (AR) learners 36 in a large educational complex in Pittsburgh. The 
AR learners were identified using criteria project members had developed from their 
professional experience: expressed hostility or overt negativity, overt anxiety about 
joining the programme, obvious uncertainty about the programme’s value, evident 
lack of commitment to staying in the programme, anxiousness expressed in body 
language and/or a desire to cut the initial interview short. The assessments were 
counter-checked by another project member and further exploration of the ARs’ 
schooling experiences was undertaken (using the Prior Schooling and Self- 
perception Inventory) on the assumption that negative school experiences increased 
the likelihood of withdrawing from the programme. They were also given the Witkins 
Embedded Figures Test to measure need for acceptance by peers, co-workers and 
friends. Participants were then randomly referred to a control and three treatment 
groups. The treatment groups had one of three strategies: a conventional classroom 
setting, but with considerable support provided by the teacher and a counsellor; small 
group tuition (4-6 learners); or one-to-one tuition. 

After three months, none of the control group was retained, 20% of the 1:1 group, 
40% of the classroom group and 60% of the small group remained. The authors 
concluded that ARs can be identified reasonably accurately at enrolment, 37 that all 
three of the intervention strategies work better than mainstream classrooms (a format 
they point out is associated with what has failed for these learners in the past) and 
that small groups may well be the most effective tuition grouping for increasing 
retention. 

Methodologically, this is not a sophisticated study, but it is a good example of 
practitioners working closely with a researcher to pilot and evaluate a number of 
strategies to overcome an issue. 



36 The original goal of 40 AR learners was not possible within the constraints of the programme 
operations. 

37 An observation also reported in Morrow et al.’s (1993) study. 
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A study to estimate the effect of persistence (the hours of instruction) on student 
outcomes (Fitzgerald & Young, 1997) examined records from 44 literacy 
programmes in 22 states, using regression techniques and a path analysis 
framework. Valid and matched pre- and post-test scores were available for a sample 
of 614 students. CASAS was used to measure the achievement of ESOL learners 
and TABE for other learners. The hypothesis was that basic skills achievement would 
increase with greater persistence. However, this only held true for ESOL students. 
The reading achievement of other students appeared to decline after nine hours 
tuition per week. The study identified several practices that do have a positive effect 
on outcomes. For ESOL this included student participation, full time and experienced 
ESOL staff and client support services; for other students they included the use of 
individualised curricula and full-time staff. 

High retention rates are often quoted as a key feature of family literacy programmes. 
A review of family literacy programme research (Padak, Sapin et al., 2002) identified 
the following features as central to promoting retention: meeting learners’ needs 
through participatory teaching, family-based activities, on-going assessment to foster 
a sense of success, social networks, integration of services, the use of authentic 
materials and the enthusiasm of the teacher. 



3.2.4 Discussion 



Based on the studies reviewed above studies, it is reasonable to conclude that a 
minimum of 100 hours over a year is needed in order to have a realistic chance of 
improving LNL skills. Learners at the more basic levels probably require even greater 
amounts of tuition to make equivalent gains. While there is no specific data on 
intensity of tuition (i.e. hours per week), more intensive programmes are logically a 
more effective way of ensuring minimum amounts of tuition. Regular attendance may 
be as, if not more important than, hours per week for ESOL learners. 

Recruiting LNL learners is not always easy and retaining them for reasonable 
amounts of tuition is an important component of successful programmes. The limited 
amount of research in this area points to the possibility of identifying potential 
withdrawals and the value of actively working to increase their retention through a 
range of strategies such as those used in the studies reviewed. 

We know the hours available for learners vary between programmes and contexts, 
but as yet there is little data about the average number of hours of tuition LNL 
students attend in a year in New Zealand; nothing is known about the regularity of 
attendance. Systematic data has not been collected across the various contexts in 
New Zealand on how many learners drop out of tuition and when. 

The nature and extent of ‘deliberate acts of LNL teaching’ is also likely to vary 
between programmes. It may be that the skill level of tutors has much to do with 
whether explicit teaching actually occurs. It may also be that literacy gain is much 
more likely to occur in programmes that have LNL as a specific focus and where 
literacy outcomes are expected rather than where literacy support is provided to help 
learners achieve non-literacy course outcomes. 
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3.3 FEATURES OF PROVISION 

This section discusses several aspects of how provision is organised, including how 
curricula is determined, the number of learners taught at a time, the impact of 
computer aided instruction and a discussion on whether tutors need to use different 
approaches with students described as learning disabled or dyslexic. 



3.3.1 Curriculum 

The focus in this section is less on specifically what is taught and more on the extent 
to which any curricula balances external programme or vocational considerations and 
individual learner needs. 

Learner-centred curricula 

Historically in New Zealand, the content of LNL programmes was largely drawn from 
learners’ experiences, interests and demands. This learner-derived content or 
authentic instruction refers to where teaching content is taken from the everyday 
environment of the learner, rather than pre-designed and sequenced programmes 
that often rely on individual workbook completion, repetition and drills. This pattern 
developed for two main reasons: the influence of British adult literacy models in the 
formative years of the field (Hill, 1990) and the fact that very little tuition at that point 
was tied to formalised qualifications that dictated curricula content (and therefore 
enabled or required the preparation of standardised material). The one exception in 
this respect was probably the ubiquitous driver’s licence test. 

This learner-centred curriculum tradition is also reflected in the writings of adult 
learning theorists such as Knowles (1984), Freire (1971) and Brookfield (1986). 
While a learner-centred curriculum is still valued by practitioners in New Zealand, 
there is no empirical evidence of how widespread the approach now is in practice. 
With the increased incorporation of unit standard assessments into LNL programmes 
there has probably been some movement away from learner-derived content. 

This review identified few robust studies that examined how curriculum was 
established. A British study of adult literacy teachers (G Brooks et al., 2001) found 
that while they used both commercially published materials and resources they had 
developed themselves, they used the published material less frequently. Authentic 
instruction is less common in ABE teaching in the United States, where there is a 
dominance of commercially supplied workbooks, readers, and textbooks, especially 
in relation to the GED. 38 

One study is particularly noteworthy in relation to curriculum (Purcell-Gates et al., 
2002). 39 The researchers examined two dimensions of adult literacy instruction - the 
‘authenticity’ (or ‘real life’ nature) of the activities and texts used in the teaching and 



38 ESOL teachers are more likely to use environmental print and pictures or realia’ (objects from 
students’ lives such as pieces of clothing, household objects etc) to get the meaning of English words 
across or to start conversations. 

39 This study (which is methodologically more sophisticated than most) is also reported as a NCSALL 
study (Purcell-Gates, Jacobson, & Soler, 2000). 
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the degree of collaboration between teachers and students - and changes in the 
literacy practices of the students, such as the types of texts read and written. In this 
way, the study explored the theoretical claim that some types of instruction are more 
effective than others. But rather than rely on changes in learner behaviour being 
measured by criterion-referenced tests, the researchers assessed the outcomes 
directly on what they call “the ultimate end-goals of all literacy instruction - actual 
reading and writing practices of the students” (p. 73). Their measure is therefore 
based on the practice of reading and writing, rather than the ability to read and write 
as measured by tests. 

The study was descriptive and correlational. Multiple methods (a teacher 
questionnaire, researcher observation and student interviews) were developed 
through extensive piloting to classify instruction across the two dimensions under 
study (authenticity and collaboration). These two dimensions were then able to be 
used to classify the classes (ranging from ‘highly teacher directed’ to ‘highly 
collaborative’ and ‘from highly authentic’ to ‘highly school-only’) and measured 
against changes in the literacy behaviours of 173 students. 

The study concluded that there was no relationship between the degree of 
collaboration and changes in literacy behaviours. On the other hand, the study did 
find a positive relationship between those engaging in real life, authentic activities in 
the classroom and changed literacy behaviours. 

The results document that it is indeed beneficial, relative to the purposes of 
adult literacy instruction, to incorporate materials and literacy activities in 
the instructional program that reflect real-life texts and purposes for reading 
and writing them to the greatest degree possible (p. 91). 

Furthermore, because the study controlled for the literacy level of the students, the 
authors concluded “there is no reading ability threshold at which this type of practice 
is inappropriate” (p. 90). 

While the authors were clear about the limitations of their study (e.g. using volunteers 
rather than random selections 40 and correlational rather than experimental), they 
believed that “this type of design is close to being the best that researchers can do 
with this population of adult learners” (p. 88). The authors claimed that: 

This is the first time that research has documented this for outside-of-school 
contexts, despite the widespread belief among academics that this type of 
instruction is best practice and despite the considerable lip service given to 
this principle by practitioners (p. 90). 

The earlier related study that established the methodology (Purcell-Gates et al., 
2000) included an examination of whether the degree of collaboration between the 
learners and teachers affected changes in reading and writing behaviours. While they 
concluded that it did not, they also cautioned that this finding may be to do with how 
they operationalised and documented the collaborative dimension. They believe 
collaboration needs further study, rather than concluding that it is not influential - 
especially as collaboration is such a strong component of much LNL philosophy. 



40 It is also interesting to note how difficult they found it locating genuinely authentic/collaborative 
classes; actual observation of classes often failed to match the teachers’ perceptions of the classes. 
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Integrated or embedded provision of LNL 

The concept of concurrently developing LNL skills and other competencies is an 
approach adopted by vocational education and training delivery in Australia (Watson, 
Nicholson, & E, 2001). It is also increasingly common in the UK 41 . Integrated literacy 
has been advocated by Skill New Zealand/TEC over recent years for TO and YT 
courses. 

There appears to be very little outcome-focused research related to embedded or 
integrated literacy provision at this point. 42 A very brief report of preliminary findings 
from a study currently underway in England and Wales (Clary, 2003) suggests the 
balance of basic skills content increases as courses progress; that planning is 
particularly important to ensure specialist literacy and subject specialists can both 
jointly prepare beforehand and liaise during programmes; that integrating or 
embedding LNL is rewarding, but time-consuming and challenging. An ‘experts 
seminar’ of 105 experienced practitioners called together at the start of the 
programme (Clary, 2002) recorded a number of factors for success that practitioners 
believed had been established from existing programmes: 

> that integrating literacy had to be seen as part of a ‘whole organisation’ 
approach 

> that learners’ needs had to be central 

> that there had to be mechanisms to recognise the LNL needs that may 
emerge for the subject specialist tutors themselves and appropriate training 
provided for them 

> that not all learners in any course might have LNL needs and this needed to 
be taken into account in planning 

> that the process takes time and additional staffing and is costly. 

It is worth noting that in England and Wales, funders require that achievements in 
both the subject area and LNL be recorded in embedded programmes. 

Discussion 



The findings that authentic instruction leads to learner gain will strike a chord with 
many practitioners, who believe that real-world literacy activities provide the best 
platform for learner engagement and motivation. It has yet to be determined to what 
extent practice in New Zealand actually builds in real life literacy events and 
situations. 

There has yet to be any substantive independent examination of how the process of 
integration of vocational and literacy skills takes place in New Zealand contexts, nor 
any in-depth analysis of whether the approach results in greater LNL gain for 
learners than LNL-specific programmes. 



41 An Embedded Literacy Portal has just been has just been developed to assist UK tutors to deliver 
LNL competencies embedded within other formal courses. 

http://www.dfes.aov.uk/readwriteplus/embeddedlearning/cfbtaeneralinfo.cfm 

42 Despite the significance of this approach in the Australian vocational education system, our review did 
not identify appropriate studies. 
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3.3.2 Class/ small group/ 1:1 teaching 

Historically in New Zealand, LNL tuition was been dominated by one-to-one teaching 
by volunteers, but over recent years (especially with the growing role of paid tutors) 
and an increase in vocational programmes, there has been much more class and 
small group provision. Workplace provision usually occurs 1:1 or in very small 
groups. Individual tuition in some ESOL contexts means there are 2-3 learners 
present. Funding sources often influence how provision is organised (e.g. ACE 
funding often requires a minimum of 6 learners per class and sometimes up to 12. 

The only study we were able to find that examined the impact of small groups versus 
1:1 was an American CT (Morrow & et al., 1993) involving three sites. Group 
students received 1.5 hours of instruction twice a week over a seven-month period, 
averaging 46 hours. Unfortunately, only nine of the initial 20 students taught in 
groups completed the programme. 43 A control group of 11 students received 1:1 
instruction for an average of 63 hours each. The study found that reading proficiency 
gain (as measured in tests) did not differ across the instructional methods (both 
groups made gains, effect size of .66), although a greater proportion of 1:1 students 
reported achieving their identified personal goals - usually furthering education 
and/or increased recreational reading. This difference however was thought to be 
primarily due to methodology problems. In keeping with these tentative findings, the 
article concludes that the study “does not support the relative superiority of either 
instructional approach in raising reading proficiency levels” (p. 25). 

Although not on group vs. 1:1 instruction, another American CT (R. E. Roberts & 
Cheek, 1994) used an intervention involving a ‘community-building’ process with a 
group of 31 prison inmates. The process involved an intensive workshop to build 
trust, communication and cohesion over two and a half days with follow-up sessions 
weekly. They were then taught using an SRA reading programme, as was a first 
control group of 34 prisoners. A second control group received no intervention of any 
kind. There were significantly higher gains for the intervention group (effect size of 
.38), although it is difficult to envisage implementing this type of intervention outside 
the prison environment, as the need to develop trust in this context is abnormally 
high. The programme also achieved considerable non-literacy outcomes in relation to 
health, substance abuse and general rehabilitation. 

The ESOL literature (see Section 3.6) for more detail suggests that group rather than 
1:1 teaching is necessary for effectiveness, to ensure ESOL learners have 
opportunities for real-life related practice in oral English; the social practice of English 
is an important pre-requisite in effective teaching. 

Discussion 



The research review did not reveal any findings on the effectiveness of one size of 
provision over another. There are many variables to take into account, including the 
skills of teachers, whether there is constant turn over of students in groups and hours 
of tuition. 

As yet we don’t have national data on the proportions of learners that receive class, 
small group or individual tuition. As more is known, we can then consider researching 



43 Although the authors claimed that there were no significant differences between those who completed 
and those who withdrew. 
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which types of provision seem more effective for what types of learners. There is 
some evidence that ESOL learners may do better in groups. 



3.3.3 Computer-based LNL 

The terminology in this area is diverse, inconsistent and confusing - a feature that is 
seen as reflective of the pace of development in its applications (Clarke & 
Engelbright, 2003). Terms commonly used include telematics, IT or ICT (information 
technology or information communications technology - usually a broader term that 
includes computer systems and may also include non-computer technology such as 
videos), CAI (computer-assisted instruction), CAL (computer-assisted learning), CBL 
(computer-based learning), CMC (computer-mediated communication) and ILS 
(integrated learning system). There is little consistency of usage of these terms in the 
literature or everyday parlance. CAI or CAL are probably more commonly used in the 
New Zealand schooling sector (Parr, 2003), and although ICT is probably the most 
common term in much LNL literature (especially in Britain), we used CAI most often 
in this report as it tends to be the term in the studies reviewed. 

The diversity of technology involved and how it is used in educational programmes is 
also considerable. First generation CAI programmes were ‘drill and practice’ with 
limited interaction or feedback and where the computer was basically a typewriter. 
Second and third generation programmes have seen an increase in the amount of 
interactivity, greater access to information and learning options, as well as linking to 
the Internet and other forms of technology. While these subsequent generations of 
CAI are undoubtedly sophisticated and much improved on their earlier counterparts, 
the time lag of adopting new systems and/or software (usually dictated by cost) 
means that most CAI in use is still relatively unsophisticated. 

Similarly, the specific software used in CAI varies considerably and there is 
considerable turnover in their currency. The studies in the review involved an 
extensive variety of software packages. While some programmes such as PLATO 
are mentioned in a number of studies, others appear to be ‘one-offs’ or have since 
disappeared. This rate of turnover is one of the reasons that pre-1990 studies were 
not included in this review. 

As yet, there is not much information about the extent of usage. One study in 1994 
(Hopey, Rethmeyer & Harvey- Morgan quoted in Lavery et al., p. 182) reported that 
two-thirds of all adult literacy programmes in the US were using computers for 
instructional purposes, but Rosen (2000) reported only one-third of ABE teachers 
were using CAI. Stites (2004) reports that few LNL practitioners incorporate the 
Internet into their CAI approaches as yet. An English study (Mellar, Kambouri, 
Sanderson, & Pavlou, 2001) reported that: 

> The predominant pattern of activity was for learners to use a desktop 
computer on their own with the tutor either talking to the whole class, or 
involved in discussion with groups or individuals. 

> Half used office software and half used direct teaching materials. 

> The ICT was mainly directed towards practice. 
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> Patterns of use varied in literacy, numeracy and ESOL contexts, but most 
teaching followed a demonstrate/independent practice model, with little 
discovery learning encouraged. 

The NRDC has just published an observational study (Mellar, Kambouri, Sanderson, 
& Pavlou, 2004) which provides useful insights into how eight experienced tutors 
actually go about using ICT into LNL teaching The eight tutors were observed 
working with 11 different classes; in total there were 24 class observations and 12 
observations looking specifically at learners. The researchers categorised tutors 
usage in four ways: technology as curriculum (i.e. learning about keyboards, Internet, 
search engines); technology as a delivery mechanism (i.e. using individualised 
learning programmes); as complement to instruction (i.e. learners worked on one 
aspect of their skills through specific educational software or application); or 
technology as instructional tool (i.e. the computer was used as appropriate in the 
same way that a white board or piece of paper might be). The findings seem to be 
similar to Mellor’s earlier study with most work being directed to practice. They did 
see ‘instructional tool’ use, where literacy and numeracy objectives were closely 
integrated with ICT and the ‘complement to instruction’ approach. However, tutors’ 
ability to integrate ICT into their teaching was very dependent on their own skills. The 
report recommends a number of ways to increase tutors’ ability: effective ICT training 
for LNL tutors, incorporating more strategies for getting learners to work 
independently; considering the changing nature of literacies in the digital age and the 
new genre of writing that is demanded; encouraging more learner control and a wider 
range of teaching styles. 

While computers and related technology have undoubtedly been an increasing 
source of motivation for adults to seek help with LNL, they have also been a valuable 
means for decreasing the social stigma of seeking help ‘Learning to use a computer’ 
is far more socially acceptable than ‘getting help with my reading or spelling’. 
Furthermore, CAI is seen as having a number of distinct qualities that make it 
appropriate for LNL including for example: the active engagement of learners; 
participation in groups; frequent interaction and feedback; and connection to real- 
world contexts. 

As Stites (2004, p. 114) points out, these qualities are consistent with what adult 
learning theorists such as Malcolm Knowles have long advocated. 44 Rachal (1995, p. 
251) also lists qualities of CAI, although the research evidence is scant to verify 
them: reduced attrition; greater self-confidence; privacy, feedback and faster 
learning. 

Research reviews 

CAI studies appear to be the most common category of research studies in LNL. Our 
search identified two reviews of American CAI literature; the first covered studies up 
to and including our 1990 cut-off date (Rachal, 1993) and then Berger (2001) 
covered studies not listed by either of Rachal’s reviews. 

Rachal’s 1993 review of experimental or quasi-experimental studies found 21 studies 
that met his criteria of adult programmes below 12 th grade that included post-tests. 
Ten of the 21 studies found no statistically significant differences between CAI and 
traditional methods, five failed to indicate significance, two showed mixed results, 



44 Wrigley 2001 contains a very useful list of general principles and indicators to assist in the 
development and evaluation of technology-based materials. 



Literature review of best available evidence on LNL teaching 




- 46 - 



three favoured CAI and one favoured traditional methods. Not surprisingly, Rachal 
(1995, p. 239) concluded that “CAI is a very useful tool with the undereducated adult, 
but it is no panacea.” 

Berger’s (2001) review examined 23 studies carried out from 1983 to 2000 and was 
less stringent on criteria (use of computers, used in ABE setting and “reported on 
effectiveness”), but it is distinctive in that he groups the studies according to the 
software package used. Again, less than half of the studies reported positive results 
for CAI and Berger makes a cautious conclusion similar to that of Rachal (1993). Like 
Rachal, Berger also severely critiqued the studies he has included, concluding (p. 
180) that “the literature base is tainted.” 

In light of the lack of clear results from the adult LNL CAI related research about 
effectiveness, we looked at the wealth of research available on the effectiveness of 
CAI in other educational sectors. Stites (2004, p. 112) reports on several meta- 
analyses of hundreds of studies from kindergarten to higher education. One 
aggregation of 12 meta-analyses of over 546 studies by J. A. Kulik concluded: 

... that students in the computer-based groups on average scored higher 
on achievement tests than control groups whose members did not use 
computers, learned more in less time, and had more positive attitudes 
towards their classes. 

A more sceptical view is that of Parr (2003) who carried out a review of the research 
literature in terms of relevance to New Zealand schools (kindergarten to Year 13, 
with an emphasis on Years 1-8). Her conclusion is that: 

... overall, the effectiveness of computer-assisted learning has not been 
conclusively demonstrated. To date, it has been shown to be less 
effective, on average, than other forms of intervention in education (p. 2). 

In particular, Parr points out that research comparing CAI with other interventions in 
relation to cost effectiveness is lacking. In other words, even if CAI is shown to be 
more effective than conventional teaching, its financial cost may well make it an 
unrealistic option. Her review also concludes that there is considerable variability in 
the results achieved with CAI (due to the diversity of contexts where it has been used 
and with studies that are independent of the companies involved tending to be more 
critical) and that results tend to be better for basic skill acquisition in maths than 
reading. Parr also reports that achievement using CAI is “inversely related to 
instructional level... student gains are greatest in primary schools ... somewhat less 
in secondary school and lowest at the tertiary level” (ibid, p 7). However, the picture 
is complicated by the additional observation that CAI may be more effective for 
learning-disabled students, which makes it difficult to make a simple conclusion from 
her review, because some LNL students may have a degree of learning disability. 

Individual CAI studies 

Research on CAI and LNL is totally dominated by quantitative studies, probably 
because it lends itself more readily to controlled trials than other interventions. For 
example, of the 36 RCTs and CTs in the NRDC review (Torgerson et al., 2004), more 
than a third were on CAI. However, we discarded more than half of these studies 
because they were carried out pre-1990 (and used technology/software that has long 
disappeared), were unobtainable, seriously flawed and/or involved learners with 
intellectual disabilities. The ones we have selected show somewhat mixed results 
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and highlight issues that need to be taken into consideration by any New Zealand 
researchers interested in pursuing studies in this area. 

Two of the studies (Batchelder & Rachal, 2000, 2000a; McKane & Greene, 1996) 
were randomised controlled trials. The former studied 75 male prison inmates, 
divided into two groups; one group received 80 hours of GED instructional material 
on computers for one hour a day over a four-week period, while the control group 
received the same amount of tuition using traditional instruction methods. While the 
CAI group results were higher than the control group’s, the differences were not 
statistically significant (effect size: .17). 

The authors report concluded: 

It should be noted that there were no statistically significant gains in either 
subject area made by either the experimental group or the control group 
during the 4-week time (p. 1 30). 

The authors prefaced their article with critiques of other CAI studies’ methodologies 
(such as high rates of attrition), which they sought to rectify in the design of their 
study. Despite their efforts, they later admit that lack of enthusiasm by the 
participating prisoners, their strong resistance to testing and the daily realities of 
prison life all made it extremely difficult to achieve the ideals expected of RCTs. The 
authors cautioned about generalising the results of CAI studies because of the 
varying quality of the programmes available, but still venture the observation that 
teacher enthusiasm may be the “single most important aspect of an effective CAI 
programme” (p. 132). This latter point is also made in a CAI study of learners with an 
intellectual disability (Nicol & Anderson, 2000). 

The second RCT (McKane & Greene’s 1996 study) initially involved 150 prisoner 
volunteers, which resulted in 51 learners in the CAI group (who also received 
traditional instruction) and 43 learners in the non-CAl group. Again, there were small 
gains for both groups, but the results were not statistically significant (effect size .16). 
As with Batchelder and Rachal’s study, the realities of prison life interfered with the 
project, meaning “the generalisability of these data is questionable due to the vast 
attrition and problems getting the information from the correctional centers” (p. 341). 

A study of 488 learners (238 in the CAI intervention group and 250 in the control 
group) in three US community colleges (Gretes & Green, 1994) involved 44-66 hours 
of tuition over 11 weeks. 45 The CAI group used a programme called READY 
(Reading to Educate And Develop Yourself) in an interactive CD-ROM format aimed 
at intermediate level readers, while the control group were taught using traditional 
ABE instruction (workbooks and conventional classroom instruction). The CAI group 
demonstrated statistically significant gains over the control group, with an effect size 
of .97. Black and Hispanic students made the greatest gains, about three times the 
gains made by those in the control group. 

A New Zealand study (Lavery, Townsend, & Wilton, 1998) of 12 learners in a 
Training Opportunities (TO) programme for unemployed workers (six in the 
intervention group and six in the control group) showed a similar effect size (.86). The 
CAI group undertook 18 one-hour sessions of Computer Curriculum Corporation 
(CCC) software, while the control group received a ‘textbook and lecture’ method 
over the same period. The authors reported (p. 188) that “in less than two months the 



45 This study is the only CAI study to meet the criteria for inclusion in Condelli & Wrigley’s (2004) major 
study of ABE/ESOL literature reported on in Section 3.6.2). 
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adults involved in computer training gained almost three years on the Burt reading 
test and more than one year on both measures of the Neale reading tests. Similar 
gains were apparent in numeracy skills, with the gain being 16 months in overall 
mathematical performance.” The control group made no improvement in their reading 
skills and a slight decline in their maths performance. The study was not able to 
randomise the learner group because of the organisational requirements of the 
programme. 

A study of 40 learners in a Canadian community college (Wilson, 1992) had three 
groupings of learners: 15 received an average of 151 hours of CAI that used INVEST 
software, covering reading (59.24 hours on average), writing (9.74 hours), maths 
(59.34 hours) and life skills (23.44 hours) over 11 weeks. Two control groups were 
used; one completed an 18-week course and the other a 44-week programme. 
Positive gains were made with the intervention group in all areas of reading and 
maths, with gains of 1.5 years in maths. However, when the results were re- 
calculated to match the length of the three programmes, the comparison groups 
made greater gains in reading skills, but the INVEST group made greater gains in 
maths (especially problem solving). 

The intervention group rated the CAI instruction more positively than the non-CAl 
groups rated the workbook activities. The authors reported that the CAI reduced the 
teacher’s role in motivating students and keeping them on-task, the students felt they 
had more control and responsibility over their learning and that the teacher had better 
information about the learners’ progress compared with traditional teaching. This 
positive rating of the CAI instruction is also reflected in the higher attendance rate of 
93% vs. 79% of comparison Group 1 . Interestingly, while the CAI group rated the 
programme very positively, they also requested additional direct instruction from the 
teacher, which confirms the frequent observation that CAI is an invaluable 
supplementary form of instruction but cannot stand on its own without a teacher. 

Another Canadian study evaluated the use of the Autoskill Reading Programme and 
PLATO over a two-year period using quantitative and qualitative data (Howard 
Research & Kysela Consulting (1997). The study included 43 learners in the 
treatment group and 40 learners in the control group; time on the computer ranged 
from two to 48 hours, with most spending 15-20 hours. Again, the report showed that 
there were no statistically significant differences between the reading performance of 
the intervention group and those receiving conventional instruction. The CAI was 
rated positively and this group “appeared more highly motivated to complete their 
academic tasks” (p. 8). 

Discussion 



Of all the factors considered in this review, computer aided instruction appears to 
encompass the greatest diversity of interventions under a single umbrella term; they 
range from the use of simple typing programmes through to sophisticated, multi- 
media-linked programmes with refined assessment and feedback components. 

While the number of studies in this area is greater than most, the evidence upon 
which conclusions can be drawn is still sparse and limited by the methodological 
flaws in the studies. It appears CAI may be marginally more effective compared with 
conventional instruction, although much better verification is still needed to 
strengthen this conclusion. Additionally, the studies often contrast CAI interventions 
with ‘traditional’ curriculum, the nature of which is not spelled out clearly and may be 
different from conventional teaching in New Zealand. 
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CAI may be more effective in maths and with learners at basic levels. There is some 
evidence for gains in reading (including reading achievement generally, as well as 
specific elements of the reading process and word recognition). 

Teachers are central to making CAI programmes work. The programmes work best 
as a supplement to other forms of instruction, rather than as a stand-alone option. 

Irrespective of its effectiveness compared with other modes of instruction, CAI is 
consistently reported as valuable for motivating reluctant or hesitant learners. While 
enjoyment is not necessarily synonymous with gains in LNL skills, it is an essential 
consideration when participation in programmes is still predominantly voluntary. Its 
increasing usage in programmes probably points to refining the research questions in 
this area. Which specific CAI programmes are effective? How is CAI best used and 
with what types of learners? How does CAI compare to using learner-centred 
curricula in our context? To what extent does learning transfer from CAI programmes 
into real life literacy activities? 



3.3.4 Learning disabilities and dyslexia 

Approximately 20% of children do not respond to the teaching and learning methods 
used in schools, many of whom later present for LNL provision describing 
themselves as having a learning disability or report having been diagnosed as 
‘dyslexic’ at school (Benseman & Tobias, 2003). Chapman, Tunmer and Allen (2003) 
state that “the term ‘learning disabilities’ (LD) has never formally existed in New 
Zealand.” Nonetheless, these terms are used by many people - amongst parents, 
teachers, and educational professionals and not the least, by learners themselves. 46 
Chapman et al. argue (p. 94) that “continuous neglect of the LD concept along with 
inadequate and inappropriate general and remedial instruction” are fundamental to 
the poor showing in the IALS study in regard to adults who self-identified as having 
learning difficulties. 

This review provides a timely opportunity to consider what the latest research offers 
by way of guidance about more effective LNL practice for these learners. We have 
drawn heavily on a recent NRDC report Developmental Dyslexia in Adults: a 
Research Review (Rice & Brooks, 2004). The review discussed key questions, such 
as: are there grounds for the condition dyslexia and can it be defined satisfactorily? 
Does dyslexia cover all adults who have difficulty with learning to read? Do adults 
with learning difficulties/dyslexia require different teaching materials, methods and 
contexts from other adults? Can different treatments of learners be justified, when 
there is no agreement as to its definition, when research into the causes of dyslexia 
are inconclusive and when there is not agreement about how dyslexia might be 
identified? 

Explanation for difficulties in learning to read has been developed from a range of 
perspectives, from socio-cultural through to physiological and genetic - each of 
which then usually point to a range of corresponding strategies or teaching methods. 



46 The federation of Specific Learning Disabilities Associations (SPELD) lobbies to have the terms 
recognised, and there is a general acknowledgement in schools of the existence of learning difficulties. 
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The latter categories of explanations are most often couched in terms of learning 
disabilities (LD) or dyslexia . 47 

Chiappe (2002) argues for the importance of the concept: 

Understanding the mechanisms that underlie dyslexia can have important 
implications for the intervention provided to individuals who suffer from it. 
Unsupported theories may lead to inappropriate interventions that may be 
ineffective or harmful. Because we found that adult dyslexics' difficulties 
were restricted to literacy and language tasks, interventions that 
specifically address these skills would be appropriate. More specifically, 
programmes that do not explicitly address literacy and language skills 
should be avoided. For example, sensory-motor training has been found 
to enhance sensory-motor skills, but has no effect on literacy skills. More 
success is likely with traditional programmes that focus on word 
recognition, spelling and comprehension strategies. 

A British review of the dyslexia literature (Rice, 2003) challenges brain-imaging 
evidence that is often claimed to show a biological difference between brains of 
dyslexics and ordinary poor readers. Whilst morphological differences in areas 
associated with reading and activational differences during reading activities do exist, 
Rice explains how at this point we do not know if the difference is what causes 
difficulties in reading, or if it is the result of those difficulties. There is as yet no 
explanation as to why there are differences. 

Rice and Brooks’ (2004) review of research argues that the statistical findings do not 
present irrefutable evidence for differences in either kind (categorically) or degree 
(dimensionally) between dyslexic and non-dyslexic people (p. 34). They conclude: 

There is no evidence from research to support a policy of differentiating 
dyslexic from non-dyslexic students in adult literacy, numeracy and ESOL. 
We offer five main reasons for our conclusion: 

1 . Both dyslexic and non-dyslexic students need to acquire the same 
knowledge and skills in literacy and numeracy. 

2. Structured and explicit tuition is appropriate for both groups. 

3. Individual differences between students occur along many 
dimensions, while all classification schemes entail overlapping 
categories. 

4. Diagnostic protocols for dyslexia in adults cannot be used with any 
confidence either to ascertain the causes (as opposed to the 
symptoms) of literacy or numeracy difficulties or to predict the 
outcomes of interventions. 

5. The construct of developmental dyslexia is insufficient for a 
systematic and thorough appraisal of learners’ difficulties in adult 
education. With respect to adult literacy, we also conclude that 
successful teaching is informed by the tutor’s understanding of 
‘normal’ language and literacy acquisition (p. 9). 

The review suggests that what is effective teaching for ‘normal’ readers will be 
effective for people with LD/dyslexia. This point is reinforced by educational 
psychologists in Northumberland (Northumberland Educational Web Site, 2004) who 



47 The term ‘reading’ rather than LNL is used here as the bulk of the literature refers to reading. The 
comparable term to dyslexia in maths is dyscalculia. 
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list approaches on their website that "a growing body of professional opinion 
informed by properly accredited research” have identified as “well worth trying,” 48 but 
acknowledge that most would be known to LNL tutors as effective teaching strategies 
for all learners. 

High quality teaching of literacy skills by well-informed, sensitive teachers 
who take account of the sometimes delicate self-esteem of the dyslexic 
will help enormously. Good teachers are far more influential than 
specialist schemes. The most effective educational interventions will be 
characterised by all the positive features of a quality individual education 
plan (p. 88). 

A Canadian literature review (Corley & Taymans, 2002) states that there is a paucity 
of research on teaching adults with LD, and also issues around applying K-12 
research to the adult context. However, they set out criteria for providing effective 
services for adults with LD: 

> assessment of learner’s needs, interests, skill levels, learning strengths and 
challenges 

> planning appropriate teaching interventions and selecting “accommodations 
or assistive technologies” (such as readers, tape recorders, computer 
software) which help to compensate for a disability 

> professional development of tutors and administrative support people to 
ensure they understand and employ best practice 

> people understand current reading research. 

Their description of best practice as a mix of teaching meta-cognition and learning 
strategies, with direct tutoring of sub-skills and specific learning tasks, appears not so 
different from the best practice described in Section 3.1 . 

Screening learners for dyslexic is not likely to be a standard part of initial assessment 
in LNL programmes. Rice and Brooks (2004, p. 88) emphasise the need for all tutors 
to be skilled in assessment of reading difficulties: 

The screening of adult literacy students for ‘dyslexia’ is difficult to justify on 
either theoretical or practical grounds. By contrast, the psychometric 
assessment of reading-related skills is essential and every adult literacy 
teacher should be competent in it. 

Assessing learning difficulties is seen as something different. The University of 
Auckland Learning Assessment Centre psychologists suggest that assessment of 
learning disabilities is a specialized area of psychology and requires years of training 
and experience at the post-graduate level. The assessment process normally 
requires several hours of direct work with the client, as well as additional time spent 
with parents, collecting information from schools, and analysing and integrating all 



48 

“These include: phonological awareness (emphasising the constituent sounds that make up spoken 
words), letter/sound correspondence (e.g. the sounds usually associated with individual letters and 
groups of letters), phonically regular words (e.g. ‘on’, ‘at’, ‘madam’, etc), sight vocabulary (useful for 
learning 'by sight’ words which are phonically irregular, e.g. ‘the’, ‘of, ‘women’, etc), reading to them 
(often underused - an additional copy of the text may prove a helpful optional extra), listening to them 
read (requires more skill than may at first be realised): paired/shared reading approaches are worth 
trying) and enhancing confidence and motivation (difficult to overemphasise its importance - low stress - 
no blame).” 
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the information into a coherent assessment of the problem. 49 This is certainly beyond 
the scope of a ‘normal’ literacy assessment process in most contexts. 

The conclusion of the NRDC literature review states (Rice & Brooks, 2004, p. 88): 

In our present state of knowledge, it does not appear to be helpful for tutors to 
think of some of their students as ‘dyslexics’ and of others as ‘ordinary poor 
readers’. 

Telling a tutor that a student is ‘dyslexic’ may elicit a number of inappropriate 
assumptions about the student’s problems and abilities. 

The research does not indicate that a different curriculum should be followed 
for ‘dyslexics’; the curriculum will depend very much on the needs of each 
individual student. 

The research does not indicate that ‘dyslexics’ and ‘ordinary poor readers’ 
should be taught by different methods; however, the methods promoted as 
specialist interventions for dyslexics are well-suited to be mainstream 
methods of reading instruction, which is how they originated. 

Discussion 



The literature we reviewed suggests that there may not be a great deal to distinguish 
learning disabled adults with other adults with poor literacy skills and stresses the 
need for adult LNL tutors to be highly skilled in how to assess reading and how to 
teach reading to meet the diverse needs of students, irrespective of the causes of the 
difficulties. 

In the light of this cautious approach, it is interesting to note that in England a 
practitioner-focused website ‘A framework for understanding dyslexia’ has just been 
launched by the Department of Education and Skills.50 The website provides 
information about recognising learners with dyslexia, and explanations of a wide 
range of appropriate teaching strategies and resources that teachers may wish to 
draw on. 

The findings of the NRDC study may have been more tentative than some of those 
working with learning-disabled students expected and at a workshop that 
accompanied the launch of the NRDC study, Brook took care to emphasise what the 
report does and does not say, 51 giving greater emphasis to practitioner wisdom. 

“The report does not say that there is no such thing or condition or problem as 
dyslexia. It does say that no two researchers or experts agree on how to define it. 

The report does not say that there are no such people as people with dyslexia. It 
does say that no two researchers or experts agree on how to identify them. 

The report does not say that nothing is known about how to help adult learners who 
have dyslexia or that we might as well all go home. It does say (pp.79-87) that there 



49 Retrieved April 29, 2004 from www.psvch.auckland.ac.nz/lac/dvslexia.htm 

50 Retrieved October 28, 2004 from www.dfes.qov.uk/readwriteplus/understandinqdvslexia 

51 Retrieved January 10, 2005 http://www.nrdc.orq.uk/uploads/documents/doc 435.pdf 
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is very little even moderately reliable evidence from research on how to do this; 
perhaps much more is known from decades of practical experience. 

The report does not say that there is no point in teaching adults with dyslexia 
separately from other adults with literacy difficulties. It does say that there is at 
present no warrant in research for this - ‘there appears to be no experimental 
evidence comparing group outcomes between adult dyslexics and “ordinary” adult 
literacy learners’ (p.86). But again practitioners may well have had success with 
teaching adult dyslexics separately. 

The fact that so little can be deduced from existing research means that there is 
everything to play for.” 
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3.4 READING 

The difficulties of researching reading in adults must be borne in mind when reading 
the research. A significant percentage of learners may have learning disabilities 
(although as section 3.3.4 illustrated, identifying the exact nature and extent of this 
incidence is difficult). There are differences between native and non-native English 
speakers that may not be noted in research (see Section 3.6). Children and adults 
matched for reading abilities have different processing abilities which means that 
research based on what is known about children’s reading has to be considered 
cautiously. Finally, speed and accuracy are important when studying basic reading 
processing but are not always included in the research design (Venezky & Sabatini, 
2002 ). 



3.4.1 Overview of reading and reading difficulties in adults 

At the heart of reading is the reading process: 

The ultimate goal in reading is comprehension. Readers read a text in order 
to understand and use the ideas and information contained in it. 
Comprehension is improved when readers understand the key concepts or 
vocabulary in a text. Reading comprehension may suffer, however, when 
readers are unable to recognize individual words in a text. A reader may be 
conceptually ready to understand a text, for example, but will not have the 
opportunity to do so if he or she cannot read the individual words. To read 
individual words, the reader must know how the letters in our alphabet are 
used to represent spoken words ( alphabetics ). This includes knowing how 
words are made up of smaller sounds ( phonemic awareness or PA), and 
how letters and combinations of letters are used to represent these sounds 
( phonics and word analysis). The ability to figure out how to read individual 
words, however, is not sufficient. Readers must also be able to rapidly 
recognize strings of words as they read phrases, sentences and longer text. 
Fluent reading is crucial to adequate comprehension (Kruidenier 2002, p. 
2 ). 

The process above is not sequential - readers must do many things at once, right 
from the beginning: 

They identify words by visual memory, match sounds to letters, pull word 
meanings from context, understand sentences as complex structures, figure 
out how the whole system of written language works, obtain information about 
content, and predict both the words and the content to come (Yatvin, 2002). 52 

Adult LNL learners do not display all of these characteristics. A number of non-impact 
related research studies are summarised below in order to provide a brief overview of 
the types of difficulties adults have when reading and to suggest some appropriate 
responses. Snow and Strucker (2000) described a cluster of difficulties probably 
experienced by LNL learners: 



52 Retrieved October 1 , 2004 from http://www.pdkintl.org/kappan/k0201vat.htm . 
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ABE/ESOL learners might have partial acquisition of phonological 
awareness (reflected in decoding problems and poor spelling); fluency 
lagging behind untimed silent comprehension; vocabulary lagging behind 
years of school completion and general knowledge stalled at [upper primary 
school] (p. 35). 

The authors make the point that when oral reading is not encouraged in programmes 
because it is not seen as an ‘authentic literacy act’, specific reading problems may go 
undetected for a long time. They argue that practitioners should find out as much as 
possible about the childhood and literacy experiences of learners, as well as 
ensuring they are very informed about the reading process overall, to ensure they 
can identify where learners have stalled in their progress. In particular the authors 
stressed the need for more professional development around the teaching of reading, 
a point also brought out in Campbell & Malicky (2002). 53 

The Adult Reading Component Study (ARCS) (Strucker & Davidson, 2003) was 
designed to describe the various characteristics of learners enrolled in US 
programmes in ways that would be useful for policy makers, practitioners and 
curriculum designers. The study involved 676 ABE and 279 ESOL 54 learners (955 in 
total) from 30 literacy programmes in seven states. In a 1.5 hour-long session, 
students were assessed on 11 components of reading: phonological awareness; 
rapid naming; word recognition; oral reading; spelling; vocabulary; and background 
knowledge. 55 

The researchers identified ten clusters of students in three main groups for the ABE 
learners (low, intermediate and GED). Strucker suggests that the different 
characteristics of learners require different approaches. Low-level students need 
explicit structured teaching in phonemic awareness and word recognition. 
Intermediate level students need greater fluency, a higher-level vocabulary and more 
background knowledge in basic school subjects such as history, geography, maths 
and science so they can progress to post-school education. On the other hand, the 
more advanced GED students need content to pass their GED. 

Davidson and Strucker (2002) analysed data from 216 participants who were in the 
low intermediate and low categories of the ARCS. Interestingly, almost 25% of this 
group identified themselves as non-English speakers, yet they had not been 
identified as such in the full study. Error patterns of 45 non-English speaking and 45 
English-speaking participants were analysed further. The non-English speakers 
resemble normally developing younger readers, with skills developing evenly in both 
print and meaning, while native English speakers more resemble children with 
reading problems, whose print skills lag behind meaning skills. Non-English speakers 
made more phonetically plausible substitutions, while native English speakers made 
more real word substitutions. Learners can only substitute real words if they have 



53 Campbell & Malicky promoted the use of informal reading inventories (their version is The Canadian 
Adult Reading Assessment) as an appropriate tool to help learners and tutors. 

54 Almost 75% of learners were Spanish-speaking and more than 80% were reported to have 
reasonable literacy skills in their first language. It appeared some required teaching more oriented to 
English as a foreign language. It is unclear what proportion of ESOL learners in New Zealand at any one 
time would have similar needs. 

55 The ARCS has evolved into a website http://www.nifl.aov/readingprofiles /FT ARCS.htm . where 
practitioners can take a mini-course in reading development, download assessment tools, enter data 
from their students to identify their profile and use resources provided on the site to help plan 
appropriate teaching. 
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heard them, so this strategy is very vocabulary-dependent. When learners need to 
‘read to learn’, it is more helpful to have independent decoding strategies practised to 
the point of automaticity. For non-English speakers it is important to develop 
knowledge of less familiar phonics and pronunciation patterns, to accommodate new 
vocabulary. 

The authors suggested that when practitioners use only silent reading and oral 
comprehension questions as assessment tools in reading, the results might lead to 
students with quite different needs being inappropriately grouped together. This could 
mean tutors spend time teaching decoding to native speakers that non-English 
speakers have already mastered and teaching vocabulary to non-English speakers 
that English speakers already know. 

Campbell and Malicky (2002) investigated the reading processes of 344 ABE 
students, finding that adults at all stages of literacy development are able to make 
effective use of their knowledge as they read. Thus, programmes should ensure that 
learners’ prior knowledge is used from the beginning (in integrated programmes), 
rather than assuming that adult beginning readers need to concentrate on strategies 
for processing print first. There needs to be greater focus on making meaning for 
those who focus too much on print cues and a greater focus on processing print and 
getting information from text for those who misinterpret meaning. 



3.4.2 Findings related to dimensions of reading 

The report Research-based Principles for Adult Basic Education Reading Instruction 
(Kruidenier, 2002) is a key review of adult reading-related research, carried out for 
the National Institute for Literacy (NIFL) and The Partnership for Reading. 56 

The purpose of Kruidenier’s review was to identify and evaluate existing adult literacy 
reading instruction and to develop research-based principles and practices for 
teaching in the ABE field. The review covered all research carried out on low-literate 
post-1 6-year-olds in a variety of contexts. 57 Experimental studies published in peer- 
reviewed journals were given priority. Only a few non-experimental studies met the 
inclusion criteria, which included having a sound analytical framework, using multiple 
data collection methods and triangulation of results. Approximately 70 qualifying 
research studies were identified in the literature search based on the criteria used. 
The review showed that there was a relatively small research base and only small 
numbers of studies on any topics. 

The review is structured around the components of reading identified as essential by 
the National Reading Panel - alphabetics (phonemic awareness and word analysis), 
fluency, vocabulary and comprehension. 58 In addition, assessment and computer 
technology related research studies were reviewed because of their significance in 



56 The Partnership for Reading is a collaborative venture between the NIFL, the National Institute of 
Child Health and Development, the US Department of Education and the US Department of Health and 
Human Services to develop scientifically-based reading-related research. This review built on a similar 
one - Report on the National Reading Panel: Teaching Children to Read (NRP) - with the addition of 
topics of particular interest to adult educators. 

57 This review included research pre-1990, studies involving adolescents and learning disabilities. 

58 The NRP’s findings were controversial, having been criticised for over-statement of the findings on 
phonics and an undervaluing of factors such as student motivation (Garan, 2001 ; Yatvin, 2002). 
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ABE. Within these major topics, the research considered a number of sub-headings 
including the settings or contexts of delivery, teaching methods and strategies, and 
learner characteristics such as learning disabilities, ESOL and motivation. 

In recognition of the limited ABE research base currently available, and to give 
guidance to ABE teachers looking for ideas on how to teach adults to read where 
there is no research, the review makes tentative suggestions from reading-related 
research in the US K-12 59 schooling system. To be included, K-12 research had to 
refer to teaching that would be plausible for adults, be based on a strong body of 
evidence and relate to those K-12 students who had not followed normal-age and 
ability level reading development (because adult beginning readers are more similar 
to children who are poor readers). The review points out that K-12 and ABE differ in 
ways that may be significant in learning to read - for example, the respective age 
differences of participants, the voluntary (and sometimes less consistent) attendance 
of adults; fewer hours of teaching for ABE students and the different interests and 
motivations of children and adults. Therefore the K-12 research can only be used to 
indicate useful lines of inquiry for the future. 

Findings of different strengths were identified from the results reported in these 
studies: Eighteen emerging research-based principles and related practices for ABE 
reading teaching were identified, each of which had to be established by at least two 
experimental studies. Thirty-two additional trends in the ABE research were 
identified, related to findings from only one experimental study. Twenty-two specific 
ideas that might be used to supplement the ABE research were derived from the K- 
12 research. 60 

When reading the findings from this study in full, a striking feature (already referred 
to) is the atomisation of research - that is, how little has been replicated and how 
studies have often concentrated on only one small aspect of the reading process. 

In each of the following sections the principles are summarised, followed, where 
applicable, by trends and comments that stem from K-12 research. 

Assessment of reading 

While there has been very little research about the relationship between assessment 
and LNL learner improvement, professional wisdom leads LNL tutors to believe it is 
an essential part of the teaching process. In the same vein, Kruidenier’s review 
assumes that accurate assessment of a learner’s ability in all aspects of reading 
(both initial and on-going) is important for effective and efficient teaching. 

The limited research findings say that when measures of achievement are obtained 
for each crucial aspect of reading - alphabetics, fluency, vocabulary, and 
comprehension - literacy learners (including ESOL and reading-disabled learners) 
show diverse (‘spiky’) profiles, with different strengths across the different aspects. 
These profiles are useful for planning teaching. 



59 Kindergarten through to Grade 12 

60 Not all the principles and trends are relevant to our review, so not all are discussed in this section. In 
some cases, the findings have been re-written to make them more understandable and applicable for 
the New Zealand context. For a full copy of the report, see 

http://www.nifl.qov/partnershipforreadinq/publications/adult ed 02.pdf 
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The diversity of learner profiles suggests that one assessment measure of reading 
may not be sufficient to identify the strengths and needs of all learners. For example, 
the common practice of silent reading with oral comprehension questions will not 
elicit enough detail about learner’s reading strengths and skill gaps for the 
development of an effective teaching plan. 61 

Alphabetics - phonemic awareness and word analysis 

Phonemic awareness (PA) is the knowledge of the sounds in spoken language, while 
word analysis is a broad concept including phonics (the relationship between sounds 
and the letters that represent them), sight recognition of words, the use of context, 
knowledge of prefixes and suffixes and dictionary skills. 

A relatively large number of studies conducted over 20 years showed that adult non- 
readers have virtually no phonemic awareness ability and are unable to consistently 
perform, on their own, almost all phonemic awareness tasks. Similarly, adult 
beginning readers perform poorly on phonemic awareness assessments. The 
number and complexity of phonemic awareness tasks learners can complete 
increase as learners’ reading levels increase. 62 

Participation in ABE programmes increased learners’ PA and word analysis, and this 
gain in turn may lead to improvement in other aspects of reading such as fluency. 
Word analysis can be taught, but requires direct teaching - participation in a 
programme per se will not improve ability. 

Trends and ideas from the K-12 literature tentatively suggest: 

> systematic phonics teaching 63 may be more effective than incidental teaching; 
it may be better to provide explicit teaching on one or two PA skills than on a 
mixture. Teaching blending and segmenting may be most effective. 

> teach adults how to manipulate phonemes (e.g. to blend and segment words) 
using both sound and letters (unlike working with pre-literate children when 
sounds only might be used). 

> teaching phonemic awareness may help adults increase in their ability to read 
sight words. 

> teaching to increase fluency, by repeated reading and guided oral reading, 
helps word recognition. 

> small group teaching for PA may possibly be more effective than whole class 
teaching. 

> Too much, as well as too little, PA instruction may be ineffective (less than 
five hours or more than 18). 



61 See earlier comments from Davison & Strucker (2002) 

62 Refer to Section 3.4.1 for a fuller description. 

63 We interpret this idea to mean explicit scaffolded teaching of phonics is useful when it is assessed as 
a specific need, not that a ‘phonics’ programme per se should be taught in its entirety before other 
aspects of reading. 
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Fluency 

Fast, accurate decoding and prosody (reading with appropriate rhythm, intonation 
and expression) are important aspects of fluent reading. 64 Increasing fluency reduces 
the time spent decoding, thereby increasing the time available to get meaning from 
text. Methods of teaching fluency centre on guided oral reading (sometimes called 
shared reading) and frequent independent reading. 

The very limited amount of research said fluency was an issue for adult literacy 
learners at both beginning and higher levels. Teaching fluency through repeated 
reading strategies may lead to improved reading achievement. 65 

Trends and ideas from the K-12 literature tentatively suggest: 

> repeated reading with feedback from teachers and peers may help with 
fluency and comprehension 

> systematic phonic instruction may help fluency 

> oral guided reading procedures may lead to greater fluency. 

Vocabulary 

A large enough reading vocabulary is crucial for getting information from written 
text. 66 There has been very little research about vocabulary in the ABE sector and 
what has occurred has produced mixed results. It appears that participation in itself in 
literacy programmes may not necessarily increase ability on vocabulary-related 
assessments but the reasons for this are unclear. 

Trends and ideas from the K-12 literature tentatively suggest: 

> teaching new words from contexts important to the learners 

> encouraging wide reading, so that learners are exposed to new vocabulary; 
however whatever is read independently should be at an appropriate reading 
level so the learner can read it fluently. 

> pre-teaching vocabulary before exposure to a text improves vocabulary 
acquisition. 67 



64 Fluent reading is not the same as skimming and scanning; fluent readers take in more text at each 
fixation of the eyes on the text and move backwards or skip words less than poor readers. 

65 Also discussed by Sabatini (2002) who compared the speeds of college and ABE students. His 
findings were that slow laboured reading even when accurate was less efficient because it meant less 
reading occurred and less automaticity (accuracy and rate) developed. 

66 This is a bind for LNL learners; they have difficulty reading new texts because of vocabulary but in 
order to acquire new vocabulary they need to hear unfamiliar words and read a wider variety of texts. 
These learners need high quality vocabulary development and frequent exposure to new types of texts 
and vocabularies (Besser et al., 2004). This is even more an issue for ESOL learners. 

67 Our professional experience suggests that of all aspects of LNL work, tutors probably focus on the 
vocabulary-related aspect of reading more than others. Besser (2004) suggests that computer 
technology can help vocabulary development and that a variety of strategies have to be developed for 
vocabulary development. 



Literature review of best available evidence on LNL teaching 




- 60 - 



Comprehension 

A number of studies showed that usually, but not always, participation in ABE 
increases students’ reading comprehension. However, very little research has 
focused on how to understand changes in comprehension or what specific factors 
mean some programmes and learners achieve better results than others. 
Assessment measures of reading comprehension have been highly variable and the 
results of studies to determine whether general participation in LNL programmes lead 
to improved reading comprehension learners have been variable also. 

ABE learners have poor functional reading comprehension achievement. They can 
often recall ideas from simple texts and locate single pieces of information, but not 
combine information from longer or more complex texts. They are more likely to view 
reading as decoding. 

Explicit (as opposed to incidental) teaching of comprehension strategies may 
improve comprehension levels. There is some evidence that addressing 
comprehension within multi-component reading teaching, including PA and word 
analysis instruction may lead to improved skills. However, identifying what elements 
make the greatest impact on learning is difficult because of the large number of 
components within a study on comprehension. For example, the diagnostic- 
prescriptive approach that appeared to lead to improvement involved assessment to 
identify strengths, needs and interests, an individualised teaching programme, 
contextualised content, and the explicit teaching of reading processes. 

Students may improve their comprehension when there is additional teaching support 
through the presence of assistant teachers or volunteers. 

Briefly, but explicitly, dealing with self-efficacy and motivation may lead to 
improvement as may using contextually relevant adult appropriate material. 

Spending at least 70% of time in class practising reading and writing, including 
deliberate discussions on reading strategies may lead to improved meta-cognitive 
abilities. 

Reading comprehension may increase in longer programmes, with programmes of 
more than 50 hours (where learners attended regularly) showing greatest gains in 
comprehension. However, some studies of the links between length of programmes 
and gain show mixed results. 

Staff with more experience or training may do better at improving the comprehension 
levels of learners. 

Trends and ideas from the K-12 literature point tentatively to teaching comprehension 
strategies as part of a multi-component approach including PA, word analysis and 
vocabulary. Effective comprehension strategies from K-12 that, when used in 
combinations, have an impact on learning include: 

> teaching learners to monitoring their comprehension and understanding as 
they read 

> collaborative learning where people learn strategies together 
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> using graphic and semantic organisers 68 to help readers draw meaning from 
the text 

> using story structure, so readers learn to ask standard questions of the text - 
who, what why when - and map out events, times etc 

> question asking where learners answer questions from the teacher 

> question generation where learners pose questions about the text 

> summarising. 



3.4.3 Specific reading research studies 

There is more research on reading than on any other aspect of LNL research and the 
discussion above stresses how fundamental it is that LNL practitioners have a sound 
grasp of the reading process and related pedagogies. We have outlined a small 
number of studies to illustrate some of the issues involved in reading-related 
research with adults. The first study was selected because of the quality of its 
methodology; the second because it considered contextual literacy teaching; and the 
third because it involved the diagnostic-prescriptive approach that is purportedly 
common practice in New Zealand provision. The fourth example shows findings from 
a just completed set of case studies of how LNL tutors approached the teaching of 
reading in the UK. 

Rich & Shepherd (1993) was identified as the most robust RCT in the NRDC (2004) 
review (with an effect size of 1.14) and was also identified in the study by Condelli 
and Wrigley (2004) as having the strongest methodology of the studies they 
analysed. The study investigates the effects of explicitly teaching two components of 
reciprocal teaching to enhance comprehension - self-questioning (making up a 
question on the main idea) and summarising information. 69 

The study involved 90 adults enrolled in two adult education programmes in New 
York City (a business school and an ABE programme). Participants were randomly 
assigned to one of five groups, a group to receive tuition in self-questioning, or in 
summarising or in both (full condition), plus a control group that received teaching 
materials only and a second control group with pre- and post-assessments only. 

The participants in the study were taught the strategies in small groups for six 45- 
minute sessions spread over 18 days. Thus the intervention lasted only 4 Vfe hours in 
total. The materials were taken from primary level science and social studies 
textbooks. In the self-questioning group, students were taught to pose one to three 
questions about each passage, while in the summarising group, students read two 
paragraphs of text and then made summarising statements. Initially the teacher 
modelled the strategy and then over subsequent sessions students were encouraged 
to take over and work collaboratively practising the strategy. 

Participants were post-tested by reading a passage and recalling it orally, and 
answering multiple-choice comprehension questions. Participants from the total 



68 For example, recognising the use of varied headings, or using different conjunctions to show cause 
and effect (because, since, if) or time sequencing (after, since, until, before). 

69 The other usual components of reciprocal teaching are clarifying ideas in the text and predicting what 
might happen next. 
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condition group (summarising and self-questioning) did better on comprehension and 
recall than the control groups. On the question task, participants in the full condition 
and self-questioning groups did better than the control groups but on the recall task 
the full condition group showed more improvement. Therefore, self-questioning 
appeared to lead to more improvement in comprehension than summarising. 

Rich and Shepherd make other important observations. First, this study showed that 
direct teaching improves comprehension. Secondly, reciprocal teaching’s efficacy as 
a method has been noted in other child-based research. 70 Thirdly, there are some 
indications that summarising strategies may take longer to learn. Despite 
improvements from pre- to post-test, participants’ scores improved overall, scores on 
average were low on completion of the training. Finally, the study probably did not 
allow enough time either for learning the strategies or for consolidation of the skills 
involved. 

The value of Rich and Shepherd’s study is to demonstrate that a teaching strategy 
that is well-established in the teaching of children may also be applicable and 
effective in teaching adults and that further research is warranted. 

Dirkx & Crawford (1993) aimed to assess the value of contextualised content in 
reading teaching in a prison (the authors noted the difficulty of finding a meaningful 
and educationally relevant context in prisons). The study participants were 18 male 
prisoners; the control group of nine continued with their current studies while the 
contextualised programme replaced the existing adult education programme for 
those in the experimental group for part of each instructional day. A commercially 
available science programme was adapted to include areas of specific interest to 
inmates such as an experiment showing the effect of acid rain on plant life, 
conducting a recycling programme, growing an indoor garden and studying the five 
senses of the brain. 

The programme ran two hours daily for eight months (although total hours were not 
stated, this may have meant approximately 320 hours). Data on reading was 
collected from pre- and post-testing of reading, observations of reading behaviours 
and semi-structured interviews with participants. 

The study reported that both the control and experimental groups made modest 
improvement in their reading, but the NRDC analysis says the effect sizes were not 
clear because not enough data was provided. Nonetheless, the study reports 
dramatic differences in behaviour between the two groups with the experimental 
group engaging in group-based active learning strategies and with a much higher 
attendance rate than the control group. 

While the prison context may not be readily applicable, the research approach is 
interesting because it grapples with integrating literacy into another body of 
knowledge which is a key consideration for TO and YT programmes. 

Cheek & Lindsey (1994) compared the effects of using a diagnostic-prescriptive 
approach to teaching reading (based on initial assessment information and using an 



70 In the recently published New Zealand handbook on effective literacy practice (Ministry of Education, 
2003b) reciprocal teaching is described as effective in improving the comprehension and critical thinking 
of fluent child readers, rather than as a general teaching strategy. 
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eclectic range of strategies including language experience 71 and individualised 
teaching) compared with a ‘traditional’ programmed approach that involved 
predominantly structured commercially available teaching materials and computers. 

In the study 71 learners were randomly assigned to either the diagnostic/prescriptive 
or control groups. Five participants did not complete the programme, leaving 33 in 
the experimental and 38 in the control groups. Participants were taught one hour 
daily for two months, with the amount of provision being therefore approximately 40 
hours in total. 

Two versions of a standardised diagnostic test were used as pre- and post-tests for 
phonetic analysis, structural analysis, literal comprehension, inferential 
comprehension and total comprehension. 

Compared to those taught using the traditional programme, adults taught using the 
experimental method had significantly higher inferential and total comprehension 
scores, but statistically similar word identification and literacy comprehension scores. 
It may be that the experimental scheme focused more on meaning making, while the 
traditional programme emphasised repetition, word analysis and literacy 
comprehension. 

The fourth study (Besser et al., 2004) involved 27 two-hour observations of teaching 
and learning in 21 sites from six providers. This project also involved 54 tutor 
interviews, and detailed analysis of the work identification, comprehension and 
phonological awareness skills of 53 learners. 

As a group study participants had difficulties in word identification, comprehension 
(both explicit and implicit), phonological awareness, decoding and spelling. Learners 
had spiky profiles - with fewer difficulties on word identification and more on spelling. 
Errors did not correlate necessarily i.e. large numbers of oral reading errors did not 
necessarily correlate with poor comprehension. 

The learners’ reading profiles fell into three categories (complementing Strucker and 
Davidson’s findings): competent readers who may have been attending to improve 
their writing and who may have needed an ‘English’ course; a larger group having 
problems with phonological awareness and spelling and who might have benefited 
from close attention to those components specifically (in a 4-6 week focused course); 
a third and larger group with ‘spiky’ profiles - not all were having difficulties in all 
areas and may have responded to specific teaching across all sub-skills. 

The researchers observed that not a great deal of intense, focused reading teaching 
took place and suggest that learners may make more progress if more quality 
teaching of reading went on. They also observed that although many learners had 
difficulties with phonological awareness, most phonics teaching was done on the 
spur of the moment and tutors did not always have an adequate grasp of phonetics 
themselves, leading to inaccurate teaching. Teachers were teaching word 
identification and decoding strategies but not intensively and the strategies taught did 
not cover the range of difficulties demonstrated by learners. 



71 An activity where the teacher acts as transcriber of a learner's speech in order for the learner to then 
be able to read their own words. The process empowers learners to express themselves, as it frees 
them from having to focus on spelling, grammar etc. 
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3.4.4 Discussion 



There is a strong body of evidence from research, backed up by the judgement of 
researchers and practitioners, that tutors need to be well trained in the reading 
process. Specifically, tutors need to be skilled enough to observe learners reading 
and understand what they are seeing and to know how to teach to overcome the 
difficulties they observe. This is congruent with the first recommendation of the New 
Zealand Literacy Experts Group Report (1999), which called for more attention to be 
paid on the development of word-level skills and strategies in beginning reading 
instruction in schools It also aligns with the findings of the trials of the draft Adult 
Literacy Achievement Framework, which highlighted the need for literacy tutors to 
have more skills and knowledge about teaching reading. 

Another strand of findings points to the need for explicit teaching of all components of 
the reading process, in order that learners might make gain. This requires both tutors 
skilled at teaching, plus programme structures that enable ‘deliberate acts of 
teaching’ to take place. 

In relation to research related to the components of reading, the strongest individual 
finding appears to be the efficacy of reciprocal reading. However, this is only from 
one study. There are a number of other strategies identified in the reading section 
that are worth investigating as research projects, including comprehension strategies, 
ways to develop fluency and automaticity and the place of oral reading in LNL 
programmes. 

Learners need different types of teaching, depending on the needs identified through 
robust assessment processes, which need to include oral reading. 

LNL practitioners have been very cautious in looking to school-based research for 
guidance. However, it would be useful to consider the tentative suggestions here 
regarding the potential usefulness of K-12 findings in the light of New Zealand 
school-based research. In addition, school based researchers could be drawn on to 
improve the quality of LNL research methodologies. 

Despite the number of studies on alphabetics, it is important not to over-emphasise 
the significance of their findings in the absence of research into other components, 
for example comprehension and the importance of motivation and context in enabling 
learner achievement. It is particularly important to emphasise that any teaching of 
phonics must relate to an identified need. 

It seems that most of the research reviewed has been undertaken in programmes 
that have explicit structured LNL teaching as their prime focus; therefore it is not 
clear to what extent these findings will translate into programmes where literacy gain 
is but one of the outcomes sought and where tutors may not be literacy specialists. 
More research is required about how reading skills can feasibly be enhanced in 
integrated programmes here in New Zealand. 



Literature review of best available evidence on LNL teaching 





- 65 - 



3.5 WRITING 

This section begins with a brief overview of the writing process, before considering 
the very limited outcome-related research available into how adult literacy learners 
develop as writers, and what teaching methods are effective in developing writing 
skills in LNL learners. 



3.5.1 The writing process 

Traditionally, teachers of writing were concerned with the end product. From the 
1970s on, research began to concentrate on understanding the cognitive process of 
writing. Flower and Flayes, early proponents of this approach (quoted in Gillespie, 
2001, p. 66) theorised that the writing process has three key components: planning - 
deciding what to write and how to say it; text generation - turning plans into written 
text (through handwriting, word processing or scribing); revision - improving existing 
text. 

These three processes do not take place in a linear fashion but are iterative, 
changing as required by the writer. Publishing is sometimes seen as a fourth stage of 
writing (Ministry of Education, 2003b). In the LNL context, this might mean presenting 
a finished product to the intended audience (e.g. sending a letter to a child’s teacher) 
Student writing may be a key publishing activity in some programmes - to both have 
their ‘voice’ heard, and also to provide authentic, adult-appropriate reading material 
for themselves and other learners. 

Process writing has been very influential but subsequent theorists have focused 
more on the significance of social, political and cultural factors outside the individual 
that impact on the relationships involved that are prompting the writing, the purposes 
of writing and ultimately what is written. These newer theories place the context of 
writing more centrally than the process. Genre based pedagogy offers learners a 
more ‘explicit and systematic explanations of the way language functions in social 
contexts (Hyland, 2003, p. 18) and argues that what is considered ‘good’ writing will 
vary from one community context to another. These approaches pay more explicit 
attention to teaching the structure of target text types than process writing. 



3.5.2 Issues raised by research 

A comprehensive discussion of research on writing related to adults is found in 
Gillespie’s chapter in the second Annual Review of Adult Literacy and Learning 
(Gillespie, 2001). She provides an extensive overview of the study of adult writing, 
including selected findings from child-based writing research, writing in ESOL 
contexts, as well as a small number of studies on adult writers. 72 



72 Including quite a few studies of individual learners only - perhaps reflecting the fact that there is very 
little organised provision focused solely on writing. Most writing is taught alongside other LNL 
components, making it difficult to locate reasonable numbers of subjects for research in this area. 
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Gillespie discusses a number of issues that could have a significant impact on the 
effectiveness of any given teaching strategy. These are: the degree of frustration 
learners experience when they are unable to express themselves in writing; the 
negative association of schools with writing; the links to reading and spelling (most 
poor writers do not have phonological awareness, for example), the concern of 
learners with lower-level processes (such as grammar, punctuation, handwriting and 
spelling) ahead of the high-level processes of constructing or composing text. 

In June 2004 the NRDC published a review of research and practice about teaching 
and learning writing (Kelly et al., 2004). 73 In it, the authors present three reviews: of 
theories about how learners develop as writers, of empirical research into what 
factors enable learners to develop, and of current teaching, learning and assessment 
approaches used in the UK. 

They summarise the theoretical issues (their first review) in terms of the 
conceptualisations of writing (writing as a process, as a stimulus for cognitive 
development, as a political activity and as connected with identity); the relationship 
between reading and writing; and adult versus child learners. The implications for 
practice that Kelly et al draw from this are presented in the final section of this 
chapter, on professional wisdom. 

In their review of primary research, the authors identified 20 studies, only nine of 
which “were rated to have at least a moderately sound methodology” (p.26). The two 
large-scale studies had already been identified as being significant studies through 
our search process (G. Brooks et al., 2001; Purcell-Gates et al., 2002), and they are 
discussed below). Considering all nine studies, the authors speculate that success 
factors include: 

> teaching process-based writing, using authentic practices and ICT 

> a programme that fosters collaboration and participation, varied practice, and 
authentic tasks 

> certain student factors (high levels of qualifications and general knowledge, 
their goals matched those of the course, being young, and having little work 
experience). 



Gillespie warns that there is relatively little known about how adult learners develop 
as writers, compared with children, or how adults develop and change as writers 
within different contexts. Kelly et al concur, stating that we are not yet able to 
determine to what extent findings from research with children can be applied to the 
adult learner. 



73 This report arises from phase one of a project into effective approaches to the teaching and learning 
of writing, which was completed in December 2003. The Learning Skills and Development Agency 
(LSDA) NRDC’s partner organisation, is currently undertaking phase two of the four-year research 
project. 
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3.5.3 Large-scale research studies 

A study by Brooks Progress in Adult Literacy: Do Learners Learn? (2001) included a 
writing dimension. 74 The authors reported that many students appeared to have 
distinctly poorer writing than reading skills and that their attainment in writing (and 
reading) was uneven from one occasion to another (examples of ‘spiky’ profiles). A 
few students were described as “quite prolific writers” (p. 64) while more than half 
(56%) produced 20 words or fewer in the pre-test. The study found that while there 
were very small (but statistically significant) improvements in the students’ length of 
script and quality of handwriting, there were no improvements in the reduction of 
errors or the complexity of writing (both higher level processes that would rate as 
more important in terms of gains). 

Unfortunately, Hamilton’s (2001) critique of this study is particularly critical of the 
writing component. She describes the writing ‘test’ as “trivial evidence” (p. 7) and 
dismisses its validity, although acknowledging the difficulty of testing this skill. It is not 
clear whether the high non-response to the single question on writing in this study 
was due to problems with interpreting the question or where the non-response is 
indicative of students not attempting to answer because of their difficulties in this 
area. Putting this criticism aside for a moment, the key point to be made is that no 
student or tutor factors were found in relation to progress in writing (Kelly et al, 2004, 
p. 27). 

The only other study of note on writing was that of Purcell-Gates, Degener, Jacobsen 
& Soler (2002), 75 which looked at the influence of collaboration and authenticity 
(using ‘real life’ material) on changes in learners’ reading and writing practices. The 
authors concluded that while the degree of collaboration between students and 
teachers made no difference, the use of authentic curriculum did. 

“...writing letters that get sent to real people in the lives of students, ... 
writing notices and accounts for a newsletter that gets printed and read by 
real people ... all of these types of activities can, according to the findings of 
this study, lead to substantive changes in the ways that students create 
literate lives outside the classroom” (p. 91). 



3.5.4 Professional wisdom 

Given the dearth of sound large scale research into effective strategies for the 
teaching and learning of writing, we have attempted to identify the professional 
wisdom embedded in the two key reviews of writing research undertaken by 
Gillespie, and by Kelly et al. 

Gillespie identifies “a number of journal articles” that “have chronicled adult literacy 
practitioners’ efforts to experiment with new ways to teach writing” (p. 87). These 
articles are too limited in scope and size to be considered here, but may prove a 
useful starting point for other researchers who want to set an agenda for research. 
Tutors have written about the effectiveness of journal writing, dialogue journals, the 



74 The major components of this study (the quality of programmes and teachers) have already been 
discussed in Section 3.1.2. 

75 Already fully discussed in Section 3.3.1 
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language experience approach, families writing and illustrating their stories together, 
goal-setting through personal writing, incorporating learning styles, and from the 
ESOL context: using native language for writing, photo stories and oral histories. 

In what appears to be an analysis largely confined to the US, Gillespie advocates 
greater use of writing based on expressing learners’ experiences and opinions 
(especially in project-based instruction focused on issues of common interest) in 
preference to more rigid topics and requirements (predominantly with an eye to 
passing the GED). She also discusses three ‘promising trends’ - project-based 
instruction, Equipped for the Future (EFF) 76 (both of which encourage writing for 
authentic purposes), and technology-based communication (using the Internet). 

Kelly et al derive implications for the writing curriculum from their review of theories 
on writing. They suggest that it ought to take into account the non-linear nature of the 
process, the links between literacy skills (integrated nature of language) and the 
significance of context. They also include The New London Group’s advice that a 
curriculum be based on situated practice 77 , explicit teaching and scaffolding 78 , and 
praxis 79 . 

They go on to recommend a number of activities for teaching and learning, including 
language experience, semantic mapping as a pre-writing activity, 80 narrative and 
personal writing and research projects. They suggest that teachers encourage 
fluency rather than competence, reading aloud to discover and correct errors, critical 
discussions on the conventions of writing, collaboration and interaction between 
learners. They stress the importance of using authentic materials, and authentic 
activities such as writing for real-life purposes for a real audience, student publishing 
and journal writing. The role of the teacher should be that of a facilitator, who 
encourages and provides positive feedback, and who collaborates with their learners 
on writing tasks. They recommend that reading and writing be integrated, as they are 
often done together in everyday life. For example, teachers could use student 
writings for learning reading. 

From their analysis of the eleven studies originally excluded because of poor 
methodology, they suggest the following teaching strategies are worth further 
investigation: encouraging beginner writers to make “meaning-based revisions to the 
content of their writing rather than restricting their revisions to surface features”, 
intergenerational approaches, and encouraging learners to draw on their own 
learning experiences from other contexts “in particular by taking the risk of making 
mistakes and by ‘active thinking-trying’ “(ibid. pp. 32-33). 

Commenting on the use of ICT, both in terms of what can be implied from theories 
and the excluded studies, they state that it holds potential especially for revising and 



76 EFF is an initiative to develop adult learning standards based on the knowledge and skills adults need 
in order to carry out their roles as parents, citizens, and workers in the 21st Century. These skills include 
strong reading, writing, and maths along with other broader foundation learning-type skills. See 
http://www.nifl.gov/nifl/eff.html 

77 Situated practice: a learning environment where learners, with different levels of skill, are immersed in 
practices that have personal meaning to them. 

78 Scaffolding: explicit explanation of a strategy or skill, modelling of that element, a discussion of how 
and when it is useful and guided practice to use it in an variety of contexts 

79 Praxis: critical reflection informing practice. 

80 A visual strategy for vocabulary and knowledge expansion, when words are displayed in related 
categories. 
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editing, encouraging peer learning, finding inspiration, and creating real purposes for 
writing (e.g. emailing). 

They also comment on issues regarding theories about writing in the workplace; that 
there is a need to teach to the specific writing demands of particular duties, and to 
teach conceptual knowledge that enables the transference of general writing skills to 
the job context. 

Finally, from their own research into current teaching and learning approaches in the 
UK, Kelly et al add that learners’ goals need to coincide with programme objectives, 
and that learners should be provided with a variety of tasks and types of activities, 
and a range of opportunities to work in different modes. 



3.5.5 Discussion 



Writing is a widely researched field but there has been very little research focusing 
on writing with adult LNL students and even less that investigates the outcomes from 
particular teaching interventions with those students that might inform this review. 

IALS and most of the common assessment tools such as the TABE and the ABLE do 
not include any writing tasks and where they are available (such as in CASAS), they 
are used infrequently. Even studies mapping learners’ actual behaviours and 
perceptions of writing tasks are few and far between (Russell, 1999) and would be an 
appropriate topic for action research projects with practitioners. 

The main implication of the very limited amount of research located, much of which is 
very small scale, is that using ‘authentic’ everyday, learner-related content is likely to 
help improve writing practices of adults. Another thread of the discussion on writing 
considers that teachers need to encourage learners to think beyond surface features 
of text. A third seems to be that teachers need to promote a range of writing 
activities. 

Beyond this point, the main recommendations are to monitor studies currently 
underway, such as the large scale NRDC study described below, and to encourage 
exploratory studies - an area that may be particularly appropriate to practitioner-led 
research. 

Without a research agenda however, scarce resources could be wasted. Therefore it 
is further suggested that stakeholders in the field (teachers of writing, LNL 
researchers, writing experts from various LNL contexts including ESOL and policy 
makers) be brought together to discuss appropriate research designs that will build 
on existing research in identified areas of priority. 
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3.6 ENGLISH FOR SPEAKERS OF OTHER LANGUAGES 
(ESOL) 



Within the parameters of this literature review (which excluded second language 
acquisition research), very few studies were identified that considered the learning 
outcomes of adult ESOL learners from particular teaching interventions. The 
research that was identified tended to relate to adults who were pre-literate or had 
low levels of literacy in their first language. 



3.6.1 Overview of ESOL 

At one time, the fields of adult literacy and ESOL, although both on the margins of 
tertiary education, were considered quite independent of each other (Suda, 2002b). 
With the advent of the concept of foundation learning, language and literacy are 
increasingly seen to be related parts of the whole and have become important 
components in the general tertiary sector. However, while ESOL and literacy 
provision are related, they are not the same and there are important differences 
between the fields that should not be overlooked (Man waring, 2001; Ministry of 
Education, 2003; C. Roberts et al., 2004; Suda, 2002a). 

Differences occur because the language profiles of ESOL and adult literacy learners 
do not correspond, particularly when people are just beginning to learn English. 
ESOL learners are focused on acquiring appropriate forms of spoken English, 
vocabulary, patterns of English grammar, and the nature and cultural significance of 
various forms of spoken and written genres of English. They also want information 
about social and cultural practices of their new community (Manwaring, 2001). Adult 
literacy learners will have greater spoken fluency and an innate knowledge of how 
the English language works but may also have limited reading fluency, poor reading 
comprehension and limited writing skills (see Section 3.4.1) 

The educational profiles of ESOL and adult literacy learners may differ also. Some 
ESOL learners may have no experience of schooling and be pre-literate in their 
mother tongue. 81 Others (both migrants and refugees) may have excellent levels of 
literacy (and general education) in their first language, positive experiences of 
schooling, tertiary qualifications and professional backgrounds. Some may have had 
exposure to formal English teaching and others may be multilingual. 

Conversely, while a significant proportion of adult literacy learners also had many 
years of schooling, many have not achieved either school or post-school 
qualifications. Some will have a history of educational failure, which may affect their 
motivation to learn. Others may have had incomplete schooling so lack general 
knowledge and higher order literacy skills. 

Some ESOL learners with refugee backgrounds may be affected by past trauma, 
which impact on their ability to learn. Others with an ESOL background may have 
lived in New Zealand for a long time and have a reasonable degree of spoken 



81 Pre-literate learners either use languages that have an oral/aural tradition or their schooling has been 
interrupted or did not occur because of political or social upheaval (Shameem et al., 2002). 
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English language fluency but also have ‘fossilised’ errors of grammatical form, where 
they have stopped short of competence in the new language. 



3.6.2 Findings about the dimensions of ESOL teaching 

A recent report Identifying Promising Literacy Interventions for Adult ESL Literacy 
students: a Review of the Literature (Condelli & Wrigley, 2004) 82 provided a 
synthesis of existing research on Adult English Second Language Learners (ESL) 83 
(usually immigrants and refugees) who lack literacy in their first language and spoken 
English language skills. They need to acquire both literacy and English language 
skills at the same time. This review was intended to inform a research project 
investigating promising literacy interventions for teaching ESL learners that could be 
tested out by an experimental design. The authors describe Adult ESL literacy as 
“existing on the nexus of adult literacy, second language acquisition, first language 
literacy development and second language reading studies” (p. 2). 

Condelli & Wrigley first looked at the adult basic education (ABE) literature to identify 
studies that used literacy-related outcome measures (i.e. reading, writing, speaking 
and listening), identifying 17 studies that met their rigorous criteria. 84 None of these 
provided robust useful insights into effective ESL teaching and only two studies 
involved adult ESL students. 

The authors then reviewed the literature on second language acquisition (SLA) much 
of which is descriptive and not related to outcomes, together with that on ESL 
pedagogy research that has mainly been carried out on children. They sieved this 
material looking for studies using adults or addressing key ESL issues, grouping the 
literature into five themes, each of which is summarised below. 

There is a relationship between oral communication skills and literacy 

A number of studies point to a strong relationship between oral proficiency and 
second language (L2) reading ability. 85 That is, until learners’ understanding of 
English vocabulary and syntax is good enough to understand basic sentences and 
expressions, they are not able to draw on any reading strategies they have in their 
first language (LI). Adult ESL literacy students who had higher oral proficiency in 
English at pre-testing showed greater gain in literacy than students with lower oral 
ability. 

These findings suggest that programmes for adult ESL students need to emphasise 
building oral competence and face-to-face communication skills before, or at least 
alongside, basic literacy skills. 



82 Already referred to in Section 2.3 

83 Referred to in New Zealand as ESOL students. 

84 Ten of the 17 studies were also incorporated into the Torgeson review, three are theses we cannot 
access, two are outside our date limitation and we were not able to find information about the other two. 

85 The inter-relationship between language proficiency and literacy skills is known as the Linguistic 
Interdependence Hypothesis. 
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Connecting literacy learning to real world tasks 

There is evidence (just as there is in general adult education research) that learners 
benefit from contextualised learning. SLA research compared teaching that focused 
on oral communication and reading and writing tasks that reflect real-world 
challenges with that using a structured curriculum that focused on syntax, vocabulary 
and the morphology of a language. 86 The findings of several studies supported the 
view that a balanced curriculum of real communications tasks and structured 
grammar together with open-ended conversation are likely to result in the greatest 
gains in language for learners. Additionally, a recent study by the review authors 
(Condelli, 2003; Condelli & Wrigley, 2004) 87 showed that students in classes where 
teachers connect teaching and communication tasks to the community and real work 
materials showed greater increases in basic reading skills than those in classes 
where those things did not happen. 

Working in groups in project based learning and cooperative-learning approaches 
appears to bring about positive results, with cooperative learning being effective 
when there are group awards and individual accountability. 

These findings suggest that ESOL learners would benefit from programmes that 
maximise the opportunity learners have to connect what is learned in the classroom 
with community life and that encourage language use outside formal provision. Many 
learners only attend part-time and structures that help them communicate in L2 
between sessions are important. Given the length of time it takes people to acquire 
fluency in another language, ESOL learners need to build the confidence and 
language competence to interact with English speakers and engage with various 
forms of print, so they can continue to engage with learning over time. 

Introducing computers and multi-media into literacy teaching 

CAI appears to offer some benefits to adult literacy learners (See Section 3.3.3) and 
there is some evidence that the use of video (whether stand alone or in multi-media) 
in language instruction has particular promise because it is not print dependent. Skill 
and drill software with its structure, repetition and immediate feedback can also be 
very helpful. 

The authors make the point that other multi-media approaches such as CD-ROM or 
on-line technologies that enable self-access learning may be worth further research. 
Other approaches using technology, such as ‘virtual visits’ or ‘key-pal’ relationship 
(email pen-pals) and shared stories and pictures may also be worth exploring. 

Using the first language (LI) to support learning the second language (L2) 

The need for learners to have sufficient L2 language in order to read in L2 has 
already been discussed. However, when learners who have literacy skills in LI are 
ready to begin to transfer what they know about reading and writing across to L2, 
they may apply rules of their first language inappropriately. Some learners may have 
to be taught how to transfer. In some cases, this could mean literacy instruction in LI , 
particularly those languages with regular spelling patterns. 



86 The rules and conventions that underlie spelling patterns, for example word endings (‘ing’, ‘ed’), the 
use of apostrophes in contractions or adding ‘s’ for plurals. 

87 Discussed more fully in Section 3.6.3 
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There is a substantial body of research (although primarily related to children) 
illustrating that dual immersion or bi-lingual programmes are effective. Two studies 
specifically on ESL populations found greater gain for those in programmes that 
taught literacy in their native language than those in standard ESL programmes, 
although both studies had major problems. Nonetheless, there is some research that 
suggests it is easier to become literate in a language one knows, rather than trying to 
acquire both language and literacy at the same time. 



Improving ESL literacy through a focus on reading 

The review’s authors make the point that learning to read in a second language is 
complex, with success dependent on oral proficiency in L2, literacy ability in LI, 
motivation, personal goals, cultural attitudes towards literacy and the various 
purposes learners bring to reading. 

While there is a significant body of research about how learners acquire literacy in 
L2, very little of it has been undertaken with low literate ESL adults. However, there 
are two areas for consideration: studies show that extensive exposure to print 
improves both vocabulary and comprehension; and specific teaching of strategies 
related to vocabulary acquisition and reading comprehension (as already discussed 
relating to LNL learners generally in Section 3.4.2) appear to enhance learner gain. 

What is not clear is at what point of language proficiency those strategies would have 
applicability to L2 learners, because the research has been based on more 
intermediate level learners. It may be that teaching a strategy in LI and then guiding 
learners on how to apply it in L2 may be possible because the strategies assume a 
reading ability that ESOL literacy learners may not have. It may also be possible to 
develop extensive reading in LI , which may have beneficial effects on L2 reading. 

Condelli and Wrigley then propose that it would be worth researching whether an 
ESOL programme that “focused explicitly and systematically on both higher and 
lower level reading processing skills” (p. 41) combined with exploration of a wide 
variety of texts might result in more gain than conventional programmes that focus on 
life skills 

An NRDC publication Adult ESOL pedagogy; a review of research (Barton & Pitt, 
2003) reviewed the limited SLA research that specifically considered the types of 
adult learner found in an ESOL classroom in the UK as well as studies into ESOL 
teaching practices and some aspects of ESOL programme organisation. The studies 
reviewed were not necessarily outcome focused and the quality of the studies was 
not reported on. However, the findings reported in this review offer useful insights 
that augment the discussion above. Findings that were referred to include: 

> A study of 133 ESOL learners found that success in language learning was 
not simply a question of attitude. The age learners entered the UK (and 
started learning English) was important. 

> A study of 41 ABE/ESOL learners in three different contexts found that 
participation in collaborative learning activities enhanced learning. The group 
provided emotional and psychological support and also challenged learners to 
broaden their perspectives. 

> A small study tracking 5 ESOL learners over 6 months including out of class 
found that everyday conversation made people fluent but that not all learners 
were able to access social networks to have that conversation outside class. 
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> Another study of 40 adult migrants in five European countries found that each 
first language trained learners to pay attention to different features and events 
that transferred to the additional language and was difficult to overcome. The 
same study found that learners often had to use their new language in tense 
and emotional situations where there were asymmetrical power relations and 
posed the question how teaching situations took this into account. 

> A small study of 13 students in a teacher centred grammar class found that 
uptake (what had been learned from a particular class) was highly 
idiosyncratic with most items being taken up by no more than 3 learners; 
learners were more likely to learn from topics raised by other learners not the 
teacher. 

> A study of learners in two New Zealand adult ESOL classes found a positive 
relationship between learners initiating discussion about a language form and 
subsequently incorporating that form into their talk. 

> Studies of classroom participation suggested that social relationships in 
classes play an important part in whether and how learners participate in 
class. 

> Several studies considered how ESOL learners communicated in pairs or 
small groups or in formal learning tasks, suggesting that the language forms 
needed for these different contexts had to be explicitly taught. 

> A number of studies indicated there may be a gap between teacher and 
learner expectations (for example, with learners wanting more systematic 
teaching). Moves to make classes more participatory were affected by 
learners’ prior educational experience and were variable in impact and 
quality. 

The review then looked at ESOL literacy research studies, most of which does not 
distinguish between those with and without first language literacy. The findings that 
appear relevant to New Zealand included: 

> One comparison of an English as a Foreign Language (EFL) and ESOL 
course found that neither course met the needs of learners; the teachers were 
not trained to help those with little literacy in the first language while the EFL 
course was too basic. Furthermore, the EFL course also assumed a certain 
level of higher education among learners but not all learners had it. 

> A number of studies looked at aspects of bilingual teaching. Studies showed 
that family literacy programmes were effective for the development of English 
literacy, with some of the programmes successfully using bilingual assistants. 
Bilingual teaching is complex and many factors influence what will be 
transferred between languages. 

> A number of studies considered the training of ESOL teachers. A comparison 
of ESOL and adult literacy pedagogical practices in Australia found that 
literacy teachers drew more on adult learning theory and whole language 
approaches (including process writing), while ESOL teachers drew more on 
linguistics, including genre theory and systemic functional linguistics. Both 
used authentic text but ESOL trained teachers modelled text more and taught 
grammatical features more explicitly. A UK study suggested that ESOL 
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learners might need literacy support in both first and second languages but 
ESOL teachers were not necessarily trained in literacy pedagogy. 88 

> Studies around the intensity of provision found that learners wanted more 
hours of tuition than was available, that there was less drop out from more 
intensive courses and one study found that progress took longer than one 
intensive six month course. Another recent study found that the general level 
of provision on offer catered for beginning to intermediate learners but was 
not sufficient to increase learners’ skills to move into work or further 
education. 



3.6.3 Specific ESOL research studies 

The What Works Study in Adult Literacy ESL students 89 (Condelli, 2003; Condelli & 
Wrigley, 2004a; H.S Wrigley, 2003) was a major mixed-method research project to 
identify effective teaching practices for ESOL learners. Over two years the study 
involved 530 detailed classroom observations in 38 classes from 13 organisations in 
seven states in the USA. 

The second component of the research involved following 495 learners in two 
cohorts for nine months after they started in their ESL programme. Students attended 
classes for approximately 16 weeks - the mean of actual teaching was 129 hours. 

Learners were assessed pre-, mid- and post- teaching, regardless of whether they 
had continued with the programme, on both reading and writing tests. In order to 
ensure learners understood the tasks that were required, instructions were given 
orally in their first language. In addition, learners had a face-to-face ‘literacy practices 
interview’ that asked how they used spoken English and their reading and writing in 
any language in their every day lives. Also, learners undertook a reading 
demonstration test, which rated how they read and understood functional print 
material that ranged in difficulty from food labels (easy), flyers (moderately difficult) to 
newspaper articles (more difficult). 

The longitudinal observational data was analysed using latent growth modelling to 
relate the teaching measures to literacy gain as measured through learner 
assessments. The modelling incorporated measures of attendance and student 
variables. Findings from the latent growth modelling showed: 



Growth in basic reading (including letter-word identification and phonics) 

> Older students learned more slowly; younger ones who started with less 
reading and writing overcame that disadvantage by learning faster. 

> Students with more formal schooling started at a higher rate and learned 
faster (which appears to support the supposition that literacy skills in LI help 



88 Conversely, in New Zealand adult literacy programmes may have to cater for ESOL learners who 
have progressed on from ESOL specific classes. It is unclear how many adult literacy teachers have the 
language teaching skills to meet their needs. 

89 The full and final report on this study is not yet available on the US Department of Education website. 
However, this research is being seen as influential and is informing other research projects underway in 
both the USA and UK. 
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in learning L2). However, years of formal schooling became less important 
over time. 

> Students’ oral skills were positively related to reading (albeit with marginal 
significance) - therefore, some oral language proficiency may assist in 
learning to read. 

> Students in classes where teachers used ‘connections to the outside’ by 
using real world materials and examples, field trips and speakers in their 
teaching had more growth in basic reading skills. 

> Students with longer scheduled classes showed less growth than those with 
fewer hours. 



Growth in reading comprehension 

> The reading comprehension of students with better reading skills at the start 
grew only slowly at the beginning, but then accelerated. In contrast, students 
with little or no skills showed initial growth and then failed to improve. 90 

> Rate of attendance appeared to matter more than the actual hours of class; 
students who attended regularly improved their reading comprehension, 
regardless of how many hours they attended. Student attending more hours 
per week showed more growth in comprehension. 

> The more teachers used LI to do such things as explain concepts and 
provide directions about class activities, the faster students’ comprehension 
grew. 

> Overall, the growth of reading comprehension was very steep when students 
entered with higher reading skill, attended regularly and when teachers 
enhanced instruction by using the students’ native language. 



Growth in oral English 

> Younger students acquired skills slightly faster than older ones. 

> As with reading, students with regular attendance learned oral skills more 
quickly than those who attended less. Also, longer classes promoted faster 
growth in oral language. 

> Students in classes where more time was spent in oral communication (e.g. 
pronunciation practice, conversation practice and dialogue drills) made more 
progress than those where those things happened less. 

> The use of LI as instructional support also helped students learn faster 

> Where teachers taught concepts in a variety of modalities (e.g. reading, 
writing, hands on activities, talking about what they were reading) and 
encouraged learners to interact with each other, students made faster growth 
in oral communication. 



90 There was no analysis of this finding in the interim report we were using. 
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Growth in writing 

> Changes in students’ writing were not statistically significant. The authors 
speculate that progress in writing may take longer to emerge than their study 
allowed for, or that their assessment instruments may not have been sensitive 
enough to measure subtle gains in early writing development. 



Some of the findings surprised the authors. Although programmes were supposed to 
be literacy-focused, 60% of time was spent on language acquisition rather than 
literacy. Teachers in adult ESL literacy were not trained to teach literacy, so used 
material they would use in ESL classes for more literate students. Also, the 
alternative reading demonstration assessment they devised did not show significant 
growth. One reason for this could have been that the instruction they observed 
generally did not link to real-life materials (as in the assessment), but instead 
stressed basic skill development or general comprehension. However, older students 
tended to perform better over time in this assessment than younger, perhaps 
showing their familiarity with ‘real-world’ material. 

The teachers in the study were very homogeneous, without enough variation in 
education or teaching credentials for any variables to be analysed. 

NRDC has recently published English for Speakers of Other Languages (ESOL) - 
Case Studies Of Provision, Learners’ Needs And Resources (C. Roberts et al., 
2004). While the five studies do not look at outcomes, they examine aspects of 
provision in great depth, thus providing a very rich description of current ESOL 
practices in Britain and some of the distinctive features of ESOL learners there. The 
scope of the case studies include: 

> The role of educational provision in orientation and pastoral support for 
asylum seekers; the demands on the teacher to be the main advocate and 
support for learners as they seek to adjust to the British way of life and find 
employment as quickly as possible. 

> Heterogeneity in ESOL classes and how that diversity is produced by 
classroom practice, it is not just something that learners ‘bring along’ to class 
e.g. the use of LI support in teaching, how learners can become teachers 
when using their own language. 

> How the pedagogy and practices in ESOL and numeracy classes differ when 
both classes are linguistically and culturally diverse. In particular, the case 
study considers ESOL learners’ responses to the emphasis on individual work 
in the numeracy classes compared with the focus on talking and interaction of 
ESOL provision. 

> The selection and use of reading texts in the teaching of reading in ESOL; 
how the teaching process reflected theories of reading, the use of real life 
material, and whether the prior knowledge and life experience of learners is 
accessed through the types of texts chosen. 

> Examines a pre-employment course for advanced ESOL learners with 
professional backgrounds, which integrated advanced language teaching and 
subject knowledge. While the ESOL teachers took a multi-strand approach 
and worked with pronunciation, word meaning and taught institutional 
knowledge, subject teachers made no allowance for gaps in linguistic or 
cultural knowledge. 
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This report makes a number of key points about the nature and structure of ESOL 
provision that may be of relevance to the New Zealand context: 

> An emphasis on individualised teaching and learning that is emphasised in 
some adult literacy delivery may not support the needs of adult ESOL 
learners. Talk is work’ in the ESOL classroom and the most significant mode 
of learning for ESOL learners is through group interaction and opportunities to 
practice speaking and listening (accompanied by explicit teaching of 
grammatical form). 

> Effective teachers of ESOL employ a series of measures to support the needs 
of ESOL learners in the classroom. Mainstream teachers need to learn from 
these approaches to better support the needs of ESOL students in their 
classrooms. 

> There is a need for more pro-active cross-agency support for refugees and 
asylum seekers. ESOL teachers in most classes were juggling a number of 
advocacy and support roles to assist learners in addition to teaching, and lack 
institutional support and specialist knowledge to do so. 

> The use of everyday, culturally-specific situations to contextualise maths 
problems may act as a barrier to attainment for ESOL learners in numeracy 
classes, when they don’t have either sufficient language knowledge or 
contextual experience. 

> Learners use their other languages in concrete and strategic ways to help 
them to learn English. Teachers can facilitate this in many ways with 
strategies to encourage the use of learners’ other languages within teaching 
and learning English. 

> It may be that the involvement of learners in the planning and reviewing of 
their learning through individual learning plans is not meaningful, as language 
learners appear unable to reflect on and predict their language development, 
even when they have an advanced level of English. 



A recent New Zealand study Through Language To Literacy: A Report On The 
Literacy Gains Of Low-Level And Pre-Literate Adult ESOL Learners In Literacy 
Classes (Shameem et al., 2002) examined the use of bi-lingual tutors for ESOL 
literacy learners in our local context. The findings align with the findings earlier in this 
section, that the use of bi-lingual tutors benefits learners and enhances gain. 91 The 
project examined the literacy gain by adult ESOL learners in one-semester 
specialised literacy classes. From an initial cohort of 1 18 students (including refugees 
and new migrants), 62 students took part. Classes ran for either 2 or 12 hours per 
week; three classes had bi-lingual tutors, three had English-speaking tutors and two 
had both. 

Students completed three self-assessments over the semester, rating themselves on 
eight performable literacy functions in each scale for both reading and writing. 
Competency based tests that were congruent with the tasks in the self-assessment 
schedule were administered pre- and post-course with oral instructions by tutors. 
There were also 1:1 interviews (using interpreters) to establish learner profiles, 
classroom observations and interviews with the bi-lingual tutors. In addition, two 



91 Another recent New Zealand study (Hope, 2003) examined the roles bi-lingual tutors played in 
provision offered by the National Association of ESOL Home Tutors (NAEHTS) in the Waikato, through 
tutor interviews and observations of teaching and provides useful insights into the challenges and issues 
facing learners and tutors in bi-lingual classes but did not discuss learner gain. 
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classes were used for more in-depth case studies: a community-based programme 
involving refugee women who had had no previous schooling; and a mixed gender 
group representing a range of languages and cultures. In the latter class, all had at 
least six years of schooling. 

The findings were in keeping with earlier discussion - those with higher literacy skills 
initially performed better. Students entered with more reading skills than writing and 
left with better reading also - i.e. there was less progress overall in writing. Students 
working with bi-lingual tutors made greater gains in reading than those working with 
English-speaking tutors. Learners in two-hour classes (40 hours in total) made 
progress in reading while those in 12 hour classes (240 hours in total) made gain in 
both reading and writing. Students said bi-lingual tutoring was useful, particularly at 
the start of the programme. 

When the mean results of initial self-assessment and performance tasks were 
compared, students over-estimated their own ability. Post-testing showed greater 
congruence between students’ self-perceptions of their ability and their results. 



3.6.4 Discussion 

Over the last 30 years, there has been extensive research into second language 
teaching and learning but this body of knowledge is not yet strongly connected with 
adult literacy pedagogy in New Zealand. The ESOL and adult literacy fields are 
related, but distinct, primarily because the language profiles of learners are not the 
same, and in many cases where learners have had little schooling, neither are the 
literacy profiles. There is more relevant outcome-related research about ESOL 
learners who have no or low literacy skills in their first language. 

Programmes need to focus on building oral language competence before, or 
alongside basic literacy skills, in order to make use of any literacy skills learners have 
in their first language. In a major study, learners made more progress in speaking in 
classes where more time was spent on oral communication and when 
communication tasks were based on real-world situations. However, the research did 
not seem to make explicit what many ESOL teachers and SLA researchers also 
believe - that to be effective, real world communicative tasks need also to be 
accompanied by the explicit teaching of grammatical form. 

Learners improve their reading when using real-world texts and tasks - ‘bringing the 
outside in’. It appears computers and multi-media technology also provide useful 
language and literacy support. 

There is evidence from a number of studies, including a New Zealand research 
project, that bi-lingual teaching to explain concepts and learning tasks increases 
learner gain. Teaching specific reading strategies such as vocabulary acquisition and 
comprehension also improves gain. 

Longer classes also appear to help learners gain oral language skills. However, 
regular attendance may be as or even more important for learners than the actual 
hours attended. In one key study, ESOL learners who attended regularly improved 
oral skills and reading comprehension, regardless of how many hours of classes they 
had per week. 
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We need to know more about how these findings might apply to learners in the New 
Zealand context, particularly when those with still low levels of spoken language and 
low literacy move out into mixed mainstream provision. The skills and competencies 
required for the teaching of ESOL literacy students need to be better understood. 

Many of the studies alluded to (but did not make explicit) the need for teachers to 
have a ‘meta-language’ for describing the structure of language, in order to make 
explicit to learners what they need to know and to analyse their language errors. This 
meta-language is more evident in the ESOL field than among adult literacy tutors. 
Teachers and learners might benefit from a closer relationship (in particular, those 
learners who over time receive teaching from both ESOL and adult literacy/numeracy 
teachers. 
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3.7 NUMERACY 

Numeracy has long stood in the shadow of its dominant partner reading, and even 
the third partner, ESOL. Usually subsumed under the term ‘literacy’, numeracy 
advocates have increasingly argued for it to be identified separately in order to 
ensure that its distinctive demands and issues are given due consideration. There 
was for example, poor coverage of mathematical processes in the 1996 IALS, which 
is being rectified in the Adult Literacy and Life-skills Survey (ALL) currently underway 
in many countries, 92 including New Zealand next year. 

Specialists in this area argue that numeracy warrants particular attention because of 
an almost total lack of training for these teachers and the extremely negative image 
of the subject (Dingwall, 2000; Tout & Schmitt, 2002). Many numeracy studies report 
a concern about the poor maths skills and understandings of the teachers teaching 
these skills, let alone their numeracy teaching skills (Dingwall, 2000; Schmitt, 2003). 
Numeracy is also distinctive in that it is mostly taught by non-specialists who are 
allocated the role by default rather than choice (Coben, 2003; Schmitt, 2003). At the 
first NRDC conference on numeracy, the NRDC Director claimed that: 

Numeracy is the most pressing issue in helping adults improve their basic 
skills.... Numeracy has always been the poor cousin with greater levels of 
need met by weaker provision and less professional capacity. Yet we live in 
a changing world, with uncertainty and risk increasing the need for skills in 
using and understanding number (U. Howard, 2004). 



3.7.1 Overview of numeracy 

There is no consensus over a definition of ‘numeracy’. In a recent NRDC literature 
Coben et al (2003) set out some of the elements of the debate, without attempting to 
develop a conclusion, or consensus definition. In summary, there is debate over: 

> The conceptualisation of numeracy - for example, as a limited proficiency in 
maths, or a meaningful social practice. 

> Whether it is a sub-set of mathematics education, or vice-versa. For the 
purposes of determining what research to include in their review they 
interpreted the term numeracy to mean “an aspect of lifelong mathematics 
education” (Coben, 2003, p. 21). 

> The value of a computational versus a functional definition (the former being 
context- and purpose-free). 

> What the purpose of numeracy is (and for whom, and from whose point of 
view) e.g. is it to enable one to cope with life/work, or to participate as a 
citizen, or to be empowered, etc? 



92 For example, the English ALL study (Department for Education and Skills, 2003) showed only 25% 
achieved Level 2 or above, implying that 23.8million Britons are at or below Level 1 ; results were lower 
for numeracy than literacy. 
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> What the scope of numeracy is - what is on the list of ‘abilities’ required to be 
numerate (including areas of knowledge and skills), and at what sort of level. 

> Terminology - numeracy, mathematical literacy, statistical literacy, financial 
literacy, quantitative literacy, matheracy, mathemacy, techno-mathematical 
literacy etc. 

Given this report’s focus on teaching strategies that may impact on numeracy gain, 
we sought out a description that would explain a process of becoming numerate. We 
do not deny the significance of context in which adults use maths, or that numeracy 
is a “bridge between math and the real world” (ibid. p. 157). However, a focus on 
naming content areas and skills seemed a pragmatic approach to take for this report 
given the definitional debate. 93 

To develop a description of content areas for numeracy teaching, we summarised 
Glass and Wallace’s review of numeracy frameworks in the UK, USA, Canada, and 
Australia (2001) and found common themes: 

> quantity and number i.e. whole numbers, money, fractions, decimals, 
percentages, ratios; and skills such as computation, using calculators, 
estimation 

> dimension and shape i.e. geometry (length, width, perimeter, area, volume, 
angles) 

> pattern and relationships i.e. inter relationships of variables, algebra 

> data and probability i.e. sampling, prediction, data analysis, likelihood and 
chance, reading charts and graphs. 

This content is taught in relation to processes: 

> the communication of mathematical information 

> reasoning and decision-making 

> being able to apply mathematical knowledge and skills to solve problems 

> interpreting results and other forms of understanding mathematical 
information. 



3.7.2 Findings from literature reviews 

The literature search for this study identified five literature reviews that analysed adult 
numeracy research. Both Brooks et al. (2001) and Belzer & St.Clair (2003) were 
unable to locate any studies that identified specific factors associated with progress 
in maths. The third, by Tout and Schmitt (2002) concluded that, unlike all other maths 
education research literature, the literature relating to adults is, in addition to being 
extremely limited, primarily concerned with contextual or cultural issues. The fourth, 
Johnston’s review (2001), although containing a section on teachers, did not reveal 



93 A broad definition of numeracy (implying numeracy as a social practice) has been published as part of 
the New Zealand Numeracy Strategy in schools - “To be numerate is to have the ability and inclination 
to use mathematics effectively - at home, at work and in the community.” 
http://www.tki. orq.nz/r/literacv numeracv/num practice e.php Retrieved October 1 9, 2004 
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any studies of adult numeracy teaching that clearly set out to research effective 
numeracy teaching practices. 

The NRDC has also published a recent review (Coben, 2003). Coben et al consider 
that there has been a “flowering of reviews of research on adult numeracy and 
mathematics education” (p. 30) mentioning twelve reviews between 1990 and 2003. 
Three of those have already been mentioned above (Brooks et al., Johnston, and 
Tout & Schmitt). 94 Only one of the remaining studies stood out as possibly being able 
to add to our understanding - Watson, Nicholson & Sharplin (2001) - because it was 
relatively recent, and Australian. Watson et al focused on literacy and numeracy in 
the vocational education and training sector. They claim that “consistent and clear 
findings have been identified in the research” (ibid, p. 35), with regard to effective 
approaches to provision and identified team teaching as a recommended strategy. 
However, the only research cited in support of that is a study in which four TAFE 
teachers and four literacy support teachers reflect on their practice (Black, 1996). 
Salter and Allen (1996) are quoted not for evidence that team teaching is effective, 
but to explain how it is popular with learners and teachers. Neither of these studies 
met the quality criteria previously established for rigorous research on effective 
teaching. 

In addition to reviewing numeracy epistemologies, 95 Coben discussed current 
demands in numeracy and related issues (including the numerate equivalent of 
dyslexia, dyscalculia (op cit, pp. 106-109). The report also covered research on 
effectiveness. Interestingly, the authors found most research on adult numeracy 
generally was dominated by qualitative and practitioner-led studies. Their conclusion 
from the review (no detailed discussion is reported on individual studies) is that: 

Evidence on the impact of adult literacy tuition is sparse and unreliable. 
Detailed critical studies of adult numeracy teaching and learning are 
required, including intervention studies, before it will be possible to 
delineate good practice in the light of evidence rather than aspiration” (p. 
116). 

Both Coben and Tout and Schmitt suggest there is some value in considering the 
literature on children’s mathematics, although it must be considered cautiously. Tout 
and Schmitt draw on what they consider to be significant findings from Grouws and 
Cebulla (2000) that relate to increased student achievement occurring where the 
teacher: 

> encourages whole class discussion following individual and group work 

> allows small group learning 

> focuses on the meaningful development of important mathematical ideas 

> provides opportunities for both invention of new knowledge and practice of 
what has already been learnt 

> incorporates learner’s intuitive problem-solving methods in their teaching 

> uses concrete materials and calculators. 96 



94 Another four mentioned were published outside our field of reference; and several others relating to 
school-based research were also presented. 

95 Theories of knowledge about a subject 

96 Tout & Schmitt do not offer a critique of this piece of work; therefore, this list should be read 
cautiously. 
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3.7.3 Specific numeracy research studies 

Four of the Randomised Controlled Trials (RCT) and eight of the Controlled Trials 
(CT) reviewed by Torgerson et al (2004) included numeracy as a component. Most of 
the numeracy interventions were delivered via Computer Aided Instruction (CAI) 
programmes. Of the four RCTs (Batchelder & Rachal, 2000a; Nicol & Anderson, 
2000; Shrum, 1985; St. Pierre et al., 1995), we excluded the Shrum study due to an 
inadequate explanation of the intervention and missing data. Those that remain offer 
little explicit discussion about numeracy and therefore do not provide guidance. 

The Batchelder & Rachal (2000) study of 71 prisoners involved CAI vs. traditional 
instruction in General Educational Development (GED) material. There was no 
statistical difference between the two groups and the effect size was .17. 

Nicol & Anderson (2000) randomised 24 ‘adults with mild learning disabilities’ 97 into 
three groups (CAI, teacher-led, no teaching control). There was a positive effect for 
the CAI, but it was not statistically significant (effect size of .71). 

St. Pierre et al.’s (1995) study was of 199 Even Start (i.e. family literacy) families and 
their controls. Again, there were no statistically significant gains (effect size .3). 

Of the eight CTs in Torgerson et al, we excluded four as too dated (all CAI studies) 
and of the other four (Broughton, 1994; Irby, 1992; Lavery et al., 1998; Wilson, 1992) 
only two appeared useful. 

Lavery et al. (1998) is the only New Zealand study included in the review. Twelve 
learners in an unemployment programme received either CAI instruction or traditional 
instruction (‘textbook and lectures’) in literacy and numeracy. They were not 
randomly allocated however, and there were substantial pre-test differences between 
the two groups. Contrary to the authors’ conclusions, Torgerson argued that the 
researchers’ effect sizes were not statistically significant. Plowever Hattie’s 
calculation undertaken as part of this review is similar to that of the authors - a .86 
effect size. Therefore, we can give some weight to their finding that computer- 
assisted instruction using the Computer Curriculum Corporation programme 
‘Successmaker’ may offer an effective way of increasing numeracy skills of adults, 
and of “fostering components of self-regulated learning” (p. 188). 

Wilson’s study (1992) also involved CAI as the intervention (15 learners) with two 
control groups of 10 and 15 participants using ‘traditional approaches’. There was a 
significant positive change for the CAI group in problem solving, but also for the 
control groups in number operations. An effect size was calculated as .48. 

During the literature review we identified two other studies related to adults. Although 
not statistically rigorous and outcome-focused they are useful for illustrating aspects 
of the research approach to numeracy. A study by Johnston et al. (1997) focused on 
15 unemployed Australian youth (14-26 years); how they could learn effectively and 
the strategies numeracy teachers needed to work successfully with them. Data was 
collected through observations, questionnaires, journals kept by the researchers, and 
interviews with both the tutors and young people. While their conclusions should be 
treated as speculative, as they did not collect any verifiable outcome data, the first 
two points below are congruent with other findings of the significance of discussion 



97 Since some adults appeared to be living in supported accommodation, it may be that some of these 
adults may have an intellectual as well as a learning disability. 
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with peers and contextualised learning in LNL teaching and learning and all three 
points correspond to aspects of quality adult education: 

> Class discussion of a specific situation (or critical incident) can allow a 
sharing of different approaches, challenge the entrenched idea of ‘one right 
answer’ and foster an understanding of the factors - both social and 
mathematical - that makes some responses more appropriate than others. 

> Everyday practices like budgeting, learning to drive, paying taxes can be a 
“rich teaching resource for the excavation and development of mathematical 
concepts and processes” and vice-versa, to start with a mathematical concept 
or skill and “try to find occasions when we can weave it back into students 
lives.” 

> Gaps and misunderstanding are learning and teaching opportunities: where a 
learner doesn’t know something the tutor has an opportunity to scaffold 
between concepts known and new, or to form a bridge between contexts so 
the learner can implement what they know and can do in different situations, 
for different purposes (pp. 115-11 9). 



Colleran, O’Donoghue and Murphy (2001) developed a numeracy programme, 
through which learners could discover the processes they use to problem-solve and 
make decisions in mathematics. It was then piloted over four months with 26 long- 
term unemployed learners, aged 25-60 years. Data to evaluate the programme was 
collected through interviews during and after the pilot with the tutors, primary 
document evidence from the learners (such as completed worksheets and 
comments) and post-implementation interviews with learners. It is difficult to assess 
the quality of this study, as there is insufficient information in the report about learner 
gain. 



3.7.4 Teaching numeracy to E SOL learners 

No research was found linking ESOL-specific teaching practices and numeracy gains 
in our search of the literature. However, a number of issues were raised that are 
distinctive to this context. 

Firstly, there appears to be a relationship between learning mathematics and a 
learner’s linguistic skills (Southwell, 2001). For learners to be able to ‘do maths’ they 
need to know for example particular meanings of everyday words in mathematics, 
and technical words, to be able to form concepts. 

Secondly, Baynham and Whitfield (2001) argue that there is a significant difference 
between a learner of numeracy and an ESOL learner of numeracy. The former can 
more readily use talk to support their learning and make sense of everyday maths 
problems, whereas the ESOL learner has to “cope with learning the register of maths 
in English and the cultural knowledge required to make sense of contextualised 
maths problems” (ibid, p. 11). 

Ciancone (1996) makes the point that the ESOL tutor ought also to be aware of the 
possible differences between themselves and their learners in terms of 
understanding of mathematical symbols and methods of computation. For example, 
in the US the comma is used for writing numbers greater than a thousand and the 
decimal point for numbers as decimals, for some non-English speaking countries this 
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may be reversed. There can also be differences in the methods used for writing out 
long division computations; the operation can be symbolised in several ways. 



3.7.5 Discussion 



It is clear that the research evidence about the numeracy of adults to date gives no 
guidance about the amount of time and the nature and extent of teaching required for 
adults to make significant progress in numeracy. 

There are some indications that computer-aided instruction may be a fruitful teaching 
strategy, especially in an area where many teachers themselves lack mathematical 
understanding, let alone appropriate skills for teaching it. However, the importance of 
it must not be overstated. It may simply be that CAI can more easily be set up to 
collect information on gain than other methods, which skews the findings. The actual 
variables that results in any gain may not be clear in CAI - is it the medium of 
workbooks on screen that makes the difference, or are different skills taught via CIA 
compared to face to face teaching, or does the order in which things are taught 
differ? Some of the other trends suggested in the literature thus far - the use of large 
and small group discussions and shared problem solving - may be less easily 
facilitated using CAI. 

There are consistent, strong calls across the reviews for the development of initial 
training and professional development of teachers in numeracy - both in their 
understanding of the subject and their skills in teaching it. 

There has been little formal exchange over the years between mathematics 
educators in schools and the LNL sector and as far as we can ascertain, no joint 
research projects. In the absence of a solid research base in New Zealand, it may be 
useful to bring together the professional wisdom of the adult LNL field and schooling 
sectors to discuss issues of distinctiveness and similarities and to establish a robust 
research programme. In addition, adult numeracy researchers may do well to 
acquaint themselves with the rich body of research on children’s mathematical 
development and everyday cognition to inform their research. 

Our understanding of adult numeracy is so limited that at this point rather than 
evaluating the impact of what does take place it may be more appropriate to 
undertake insight studies that consider the range and nature of adult mathematical 
understandings, how that understanding develops and what interventions might be 
effective. 
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3.8 LITERACY CONTEXTS 

The ideal aim in reviewing research in different contexts is to ascertain their 
effectiveness vis-a-vis other forms of provision - for example, are workplace 
programmes inherently more effective than community-based programmes? There 
are very few studies that have considered this question and the few that exist are of 
poor quality. This question can only be answered by RCTs and involves difficult 
methodological issues such as sample sizes and equating sample subjects. Some 
studies have been included that compare the effects of a form of provision versus no 
intervention, but these do not differentiate the specific qualities of the provision that is 
offered. 

The related question therefore is to ask which factors appear to make programmes in 
a particular context more effective. For example, what factors are associated with 
successful family literacy programmes? These studies are more common and can 
provide some pointers about what works that may be useful for providers. 



3.8.1 Family literacy 

Family literacy programmes are distinctive in that they break conventional moulds of 
provision by involving both parents and their children as learners. 98 The rationale for 
treating the family as a ‘learning unit’ is based on a number of assumptions, most of 
which are largely untested by research (Benseman, 2004). 

The first assumption behind family literacy as a form of educational provision is that 
“greater benefits to both adult and child learners will be attained by taking an 
integrated family literacy approach than by independently addressing adult and child 
needs through separate, high quality adult literacy and child intervention 
programmes” (Padak, Sapin & Baycich, 2002, p. 29). The authors of this statement 
point out that this assumption is still largely unproven, although it is testable using a 
large-scale, interdisciplinary effort involving adult literacy, family literacy and early 
childhood professionals. In the US, family literacy has been advocated as a more 
effective form of provision than conventional adults-only programmes (Seaman, 
1992). The predominant model of family literacy (in the US) is the Kenan model, 
incorporating four elements - adult education, child education, parent education and 
parent and child together time (PACT). 

A second argument is that many children who are struggling with literacy come from 
homes where the parents themselves also have literacy difficulties and therefore 
provide a significant ‘two-tier catchment area’ for literacy programmes. There is 
evidence of a high correlation of literacy skills between generations of families. For 
example, a national longitudinal study in Britain showed that 72% of children whose 
parents had reading problems and low incomes were in the lowest score reading 
group (ALBSU, 1993). 

A third assumption is that parents, and especially mothers (Sticht & McDonald, 
1990), are not only influential in their children’s literacy development, but are more 



98 Although family literacy programmes are found in many countries, they are most common in the US. 
The most predominant source of funding is Even Start, with a 2002 budget in excess of $US250m. 
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likely to exert an even more positive influence when they are able to enhance their 
own literacy skills (Benjamin, 1993; Henderson & Berla, 1994; Mansbach, 1993). In a 
discussion about the development of learning motivation in families, Smith and 
Spurling (2001) argue that parenthood “remoulds the parents’ own awareness in 
such ways that their whole approach to motivational assessment can be altered” (p. 
54) and that parenthood is “of central importance in the whole motivation story, and 
has profound effects on wider social realities”. In other words, adults who may not be 
motivated to learn for themselves, find that the experience of being a parent not only 
increases their learning aspirations for their children, but through active involvement 
their own learning motivation increases, which in turn helps their children’s motivation 
- thereby stimulating a positive learning spiral for both generations. 

Research studies on family literacy effectiveness 

Only one study has endeavoured to compare the effectiveness of family literacy 
programmes against other forms of provision (Philliber et al., 1996). The learning 
gains of two groups of family literacy participants were compared with those of what 
were termed ‘adult-focused’ basic education programmes. In both comparisons 
(using both the CASAS and the TABE for learning assessment), the family literacy 
participants’ gains were significantly higher. However, the data from the two 
comparison programmes were totally unrelated to the family literacy programmes, 
with a different amount of tuition offered to control and intervention groups and very 
poor comparability in terms of the sample characteristics and size. For example there 
were 111 in the first family literacy group compared with 1,539 participants in the 
comparison group; in the second comparison the figures are 133 versus ‘N 
unreported’. While the authors readily admit to flaws in their methodology, they claim 
that the total consistency of family literacy participants gaining more than their ‘adult- 
focused’ counterparts provides a reasonable basis for family literacy’s superiority. 
Interestingly, they point to family literacy’s higher rates of retention" as fundamental 
to its better results - a factor that they have not attempted to control for. 

Two randomised controlled studies of the US Even Start family literacy programmes 
were included in the NRDC review (St Pierre et al., 2003; St. Pierre et al., 1995). The 
1995 study evaluated the impact on participants at sites in five different states; 199 
families were randomly assigned to either a family literacy programme (101) or 
received no instruction (98). All were pre-tested on CASAS, but only 64 family 
literacy and 53 control participants were re-tested on completion of the programme. 
The Even Start participants varied in how much instruction they received. Both 
groups showed gains, but these gains were not statistically significant; the effect size 
was .3. This study was part of a larger project involving 16,000 families, but the larger 
study was less rigorous in its design (St. Pierre et al., 1995). 

The 2003 RCT 100 incorporated a larger and more rigorous methodology involving 18 
programs in different states. Again, there was a control group (who agreed to not 
enrol in a family literacy programme, but about a third did seek ‘some form of other 
help’) and family literacy participants. Of the 469 adults, 417 were pre-tested and 
then post-tested eight months later (with a 20% attrition). This study also showed no 
significant differences between the two groups (no effect size available). 



99 High retention rates are seen as a consistently positive feature of family literacy programmes (Padak, 
Sapin et al., 2002). 

100 We were not able to locate this study; the findings reported here come from Condelli & Wrigley 
(2004). 
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Research on factors that impact on family literacy effectiveness 

A series of British studies of family literacy programmes (Basic Skills Agency, 1998; 
Brooks, 1998; Brooks et al., 1997; Brooks, Harman, Hutchison, Kendall, & Wilkin, 
1996, 1999) reported learning gains for both the adult and child participants. 
Although these studies did not compare the family literacy programmes with any 
other form of provision, they did provide a summary of the factors seen by the 
researchers as contributing to the parents’ progress (Torgerson et al., 2004). While 
many of these were similar to those quality factors identified for LNL programmes 
generally there were several specific to this context (marked *): 

> the fact that participation was voluntary 

> parents’ commitment to improving their children’s chances* 

> clear information about goals, including subsequent progression 

> clear focus on literacy/numeracy development, regarding other benefits as 
‘bonuses’* 

> nationally recognised accreditation of learning 

> careful selection of courses and staff, and quality of teaching 

> time-limited courses to focus achievement 

> support of key personnel within institutions such as head teachers 

> inclusion of talking and writing, as well as reading in literacy schemes* 

> teaching sessions for parents only and children only, as well as together* (p. 
16). 

A literature review commissioned by the Educational Resources Information Center 
Clearinghouse on Adult, Career and Vocational Education in the US identified 35 
research studies 101 of family literacy programmes from 1990 to 2001 (Padak, Sapin 
et al., 2002). The researchers identified the following characteristics of successful 
programmes (many elements appear generic to quality provision generally): 

> Staff: appropriately qualified, committed staff whose skills match the unique 
challenges of family literacy and have ready access to professional 
development. 

> Curriculum: builds self-efficacy and based on learners’ immediate interests 
and needs (‘real life’ content), taught in a flexible, non-threatening manner. 

> Instructional programmes and practices: identifies the value of learning 
journals, home visits, grounding instruction in the participants’ cultures and 
communities and collaboration between participating groups to ensure 
programme integration. 

Following an initial pilot programme, the National Center for Family Literacy funded 
1 1 programs in six cities throughout the US for two years. The evaluation focused on 
the efficacy of using family literacy services to both help assist welfare recipients in 
developing their skills and find employment, as well as developing their children 
academically and socially (Alamprese, 2001b). The process evaluation involved 
extensive site visits to interview key staff, observe programs and review program 



101 Some of those are included in this review but overall the studies were of highly variable quality. 



Literature review of best available evidence on LNL teaching 




- 90 - 



documents and participant data. The study identified a number of key factors (pp. 32- 
34): 

> A solid organisational structure, including administrative leadership, qualified 
staff, adequate facilities, the availability of the target population of adults, 
adequate fiscal resources and an understanding of the services to be 
delivered. 

> Programme co-ordination: the staff’s capacity to co-ordinate services within 
the programme with external agencies and organisations. 

> Integration of services, including the incorporation of work preparation 
activities into the adult education and parent time components. 



Looking to New Zealand research, two pilot family literacy programmes in Manukau 
City were evaluated through participant observation, interviews with learners, 
teachers and other key informants such as the children’s teachers, together with 
programme outcome data (Benseman, 2002, 2003c). The Manukau Family Literacy 
Programme (MFLP) evaluation listed the following factors as central to the outcomes 
achieved in the programme (again, there is some overlap with general elements of 
quality provision): 

> A lead agency to plan and co-ordinate the overall project. 

> A skilled adult educator who understands and copes with the multi-faceted 
demands of family literacy. 

> A programme of reasonable duration and intensity built around the four 
components of family literacy - parent education, child education, adult 
education and parent and child time together (PACT). 

> Physical proximity for early childhood and primary school partner institutions. 

> Public celebration of key events and achievements (e.g. graduations). 

> A ‘second step’ programme for the adult learners as they move on from the 
programme to maintain the “community of learning” built through their 
participation in MFLP. 

> Sustained commitment from all of the partner institutions. 

> Clear understanding between all participating institutions of their respective 
obligations and responsibilities. 

> An adult-appropriate teaching space in a central location. 

> Adequate funding to ensure all components of the programme are available. 

> Regular professional development that involves staff at all levels of 
involvement. 

> Regular and on-going management/operational meetings and 
communications to ensure smooth functioning of the daily routines. 

Another evaluation of whanau literacy programmes in community settings in New 
Zealand (May et al., 2004) did not identify any characteristics of successful 
programmes beyond general statements of being learner-centred and welcoming to 
learners, most of whom have had negative experiences at school. 
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Discussion 



Because of its Federal funding base, family literacy programmes in the US have 
been evaluated at an above average rate compared with many other types of 
programmes. Nonetheless, a review of all the evaluations of US family literacy 
programmes thus far concluded that the research base for family literacy is still in its 
infancy (Wasik, 2001). This review concluded that the results of the evaluations to 
date are generally positive, but “mixed.” Padak et al.’s (2002) research review is 
more positive about the outcomes achieved, but also argues the need for a stronger 
research base, particularly from longitudinal studies. 

The distinctive value of family literacy vis-a-vis other forms of provision is still largely 
unproven, but better retention rates and the strong motivational base of parents 
wanting to help their children are probably central to the positive outcomes reported 
in the large number of evaluations carried out, especially in the US. 

Because family literacy programmes involve multiple partners, their involvement in 
the programme needs strong and sustained co-ordination and careful programme 
integration, including support services. Other key factors such as qualified staff, 
sustained periods of tuition, security of funding at realistic levels, curricula based on 
learners’ needs and interests and so forth are not unique to family literacy, but 
common to most types of programmes. 



3.8.2 Workplace programmes 

Gaining access to learners through their workplaces is a form of provision that has 
developed very strongly over the past decade and a half, both in New Zealand and in 
a number of other countries. These programmes have been criticised for their 
functional literacy approach and the dominance of employer interests over those of 
the learner (Imel, 1998; May et al., 2004) in contrast to provision deemed to be 
learner-driven. Despite criticisms, there is ample evidence that in addition to 
achieving work-related outcomes, these programmes also achieve considerable 
‘ripple effects’ in the learners’ households and communities (Benseman, 1991, 1992, 
2000). These outcomes in both contexts have led Sticht to claim that funding for the 
programmes are ‘double-duty dollars’ (Sticht, 2000). 102 

The evidence of the effectiveness of workplace LNL programmes is summarised in a 
recent British literature review (Ananiadou, Emslie-Henry, Evans, & Wolf, 2004): 

No systematic data are available for the UK on the benefits to employers of 
investing in basic skills training. International evidence is also very limited, 
but some studies have suggested that employer-provided literacy and 
numeracy courses may raise productivity, improve the use of new 
technology in the workplace, contribute to enhanced customer satisfaction, 
save time, and reduce costs (see Bloom et al. 1997; Pearson, 1996; 
Hollenbeck, 1996; Krueger and Rouse, 1998). However, these results are 



102 The same may be said for other forms of provision and needs to be tested in research; we do not 
know the extent to which effects are limited to provision context. 
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based on a handful of research studies and must therefore be treated as 
extremely tentative, and in need of corroboration (p. 10). 103 

Two other British reviews that included research into workplace basic skills (G. 
Brooks et al., 2001; Torgerson et al., 2004) found no substantial studies of the 
effectiveness of these programmes versus other forms of provision. None of the CTs 
or the RCTs in Torgerson’s review for example was workplace-related. 

Torgerson’s review (2004, p. 17) identified factors “related to progress,” noting that 
they appeared to be based on participants’ and researchers’ judgements, rather than 
evidence. They included consultation between and commitment of all stakeholders 
and in particular the commitment of the company; subsidised costs; time allowances; 
and a suitable training ethos. They pointed out that none of these factors is related to 
pedagogy. 

A British study of 17 employers (Basic Skills Agency, 1995) across a range of 
companies developed a typology of the variety of ways in which companies were 
involved in workplace training, in order to make qualitative assessments of their 
effectiveness. The report suggested that a ‘company strategy’ model where training 
is an intrinsic component of its operations (as opposed to a problem-centred 
approach that focuses only on an immediate issue) is the most cost-effective model 
because both the company and the learners benefit from the programme. Contrary to 
the popular belief of programmes leading to ‘poaching’ by rival companies, 
participants valued the opportunity for learning and felt more loyal to the company 
providing the programme - factors also reported in other studies (Ananiadou et al., 
2004; Benseman, 1995). 

There are several reviews on workplace provision that list factors judged to 
determine effectiveness - often couched in terms of ‘best practice’. The sources are: 

> a Workbase (1999) report, based on their experience as New Zealand 
workplace providers 

> a seminar of British experts reported in Payne (2002) 

> Taylor’s (1995) study of key stakeholders at eight worksites 

> Mikulecky & Lloyd’s (1 997) study of two worksites 

> Belfiore’s (2002) report of Canadian practitioners’ findings. 

Below is a summary of their findings. None of these sources rates highly in terms of 
research quality, but they represent a reasonable summary of practitioners’ 
accumulated wisdom in this area. 



103 The authors point out that there is a large research literature on the effectiveness of general 
workplace training, but these studies are not on LNL per se. 



Literature review of best available evidence on LNL teaching 




- 93 - 



Table 1. Summary of key factors in workplace literacy programmes 



Factor 


Workbase 

(1999) 


British experts 
seminar 
(Payne, 2002) 


Taylor 

(1995) 


Mikulecky & 
Lloyd (1995) 


Belfiore 

(2002) 


Intense teaching of 
skills 






X 


X 




Supportive 

environment/culture of 
learning 


X 




X 


X 




High levels of 
commitment from 
company 




X 


X 


X 


X 


Support services 






X 






Adequate funding & 
time allocation 


X 




X 


X 




Flexible structures 


X 




X 






Curriculum related to 
context (‘real life’) 


X 






X 


X 


Working within ‘the 
grain’ of climate 




X 








Program tailored to 
local situation 


X 




X 




X 


Basic skills integral to 
workplace training 


X 




X 






Involvement of 
workers/unions 


X 


X 






X 


Needs analysis done 


X 








X 


Clear, ‘non- 
stigmatised’ 
advertising 


X 




X 




X 


Provision free, 
voluntary & in work 
time 


X 


X 






X 


Clear planning & 
guidance for learners 




X 




X 





Discussion 



Despite its growing role in LNL, workplace provision does not have a large research 
base on its distinctive features in relation to effectiveness, irrespective of the 
outcomes being measured (Payne, 2002). The research available points to key 
factors being company commitment, an environment supportive of learning, 
adequate time and funding, a real life curriculum, a programme tailored for specific 
needs; employee involvement, and in-work time provision. There is no guidance as 
yet about teaching methodologies. 
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3.8.3 Community-based programmes 

LNL programmes grew out of a voluntary, community base and this form of provision 
dominated the sector for many years. Since then there has been a steady increase in 
the number of paid teaching and administration positions in many community-based 
LNL organisations, although voluntarism remains an essential element of their 
operations. More recently some have diversified into contractual provision (such as in 
the workplace, for Work and Income (WINZ) and in prisons). Nonetheless, these 
organisations remain the backbone of provision in towns and smaller centres and 
their needs-based, learner-centred philosophy is still a strong influence on the 
sector’s operations. 

Despite the importance of community-based programmes, the research base on their 
operations is notably thin. 104 

An NRDC exploratory study of 1 1 providers (Hannon & Pahl, 2003) concluded that 
this form of provision was ‘distinctive’ in relation to three main issues: vision, 
development and particularly, delivery (with a holistic view of learning, concern about 
learning situations, quality, integrating basic skills without making them too apparent 
and achievement and progression). The study offers no evidence of the validity of 
these claims, but appears to be the preamble to a follow-up study - although its 
specific aims are not clear at this stage. 

A New Zealand study of community-based whanau literacy programmes (May et al., 
2004) included some observations about their distinctive features, including “a 
holistic approach to literacy that differs from many functional literacy work- 
based/vocational programmes, a client group that might not otherwise be reached by 
other literacy programmes and a more flexible and formative approach to assessing 
learner outcomes” among its conclusions (p. 1). Apart from their emphasis on 
providing a learning environment that is fundamentally un-school-like and non- 
threatening, the report does not offer any substantial evidence for these claims. 
There is certainly no basis for making claims of community programmes vis-a-vis 
workplace programmes as the study did not include any of the latter, nor are there 
any references to studies of them. 



3.8.4 Vocational provision 

Vocational provision (sometimes called unemployment or pre-employment 
programmes) is yet another area where methodologically rigorous research on 
effective practice appears not to have been done. 

An longitudinal study of vocational education and training effectiveness in rural 
Australia (Guenther, 2001) included a sub-analysis of literacy and numeracy 
provision. Based on ten detailed case studies, the research involved 541 structured 
interviews and 570 questionnaires of a range of stakeholders such as training 
providers, employers and community members. The study concluded that delivery of 
these services in small towns and regions is more effective when: 



Brooks et al.’s (2001) study of literacy learners in England and Wales is sometimes quoted as a 
study of community-based provision, but the majority of the subjects were from Further Education 
colleges rather than Local Education Authority programmes. It is questionable whether FE provision 
could be truly categorised as ‘community-based’. 
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> there is a supportive and continuing structure and infrastructure (including 
quality measures, adequate funding and professional development) 

> the content of training is targeted to meet individual and community needs 
(relevant to ‘real life’ and tailored to individual learning needs). 

> there is community ownership of programs and resources 

> there is trust and collaboration between providers and communities. 

Discussion 



Despite their centrality in terms of provision and philosophy, community-based 
programmes have been the focus of very little research. There are significant 
methodological challenges associated with researching in a community context: 
programmes that have multiple facets; part time or voluntary staff; a lack of research 
experience, skills or culture among staff; limited access to research funding; relatively 
small student numbers making sampling difficult; and the limited hours of provision 
per week they are able to offer. 105 

The search criteria we used did not identify outcome-related studies that robustly 
identified effective teaching practices in vocational education. 

What little research is available in this area is methodologically weak and is rather 
vague in identifying elements that contribute to programme effectiveness. 



3.8.5 Prison programmes 

There is a common perception of a link between LNL difficulties and crime - poor LNL 
skills cause people to turn to crime. There is certainly considerable evidence of a link 
(Smith 1994), but not of causation. Although prisoners were excluded from the 1996 
IALS study, there is some evidence of LNL issues in New Zealand prisons (Mudford, 
1993), although the study is now dated and was of questionable quality because of 
its poor sampling and administration procedures. LNL teaching does take place in 
prisons but may be less visible than other contexts, possibly because it comes under 
the Department of Corrections rather than educational structures. 

While quite a few of the RCTs and CTs in the Torgerson review were set in prisons 
(Batchelder & Rachal, 2000a; Diem & Fairweather, 1980; McKane & Greene, 1996; 
Meyer, 1983; R. E. Roberts & Cheek, 1994; Shrum, 1985), these studies focused on 
some aspect of LNL teaching such as CAI rather than prison programmes per se. 
Similarly, there are studies that report prison programmes achieving learner gains, 
but that do not specify particular elements of those programmes (G. Brooks et al., 
2001 ). 

A key question for prison programmes is whether participation in educational 
programmes (including LNL) helps reduce recidivism. One American review of 
research on this issue (Jancic, 1998) concluded that it does. Jancic focused his 



105 It is no accident that many LNL studies are set in prisons, where organisational variables are thought 
to be more easily controlled. 
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review primarily on educational programmes (ranging from basic skills to post- 
secondary level), rather than social or non-academic programmes such as drug and 
alcohol issues. While there are specific studies on GED preparation, post-secondary 
programmes and vocational programmes (all positive), there is none specifically on 
basic skills programmes. The value of completing a GED while incarcerated is 
positive, although those completing a post-secondary qualification (i.e. well beyond 
ABE or GED levels) have the highest success rates. 

Discussion 



Again, the evidence relating specifically to the distinctiveness of prison LNL 
programmes is lacking, and what is available, lacks rigour. Participating in these 
programmes appears to be more beneficial than not, but substantive evidence is yet 
to be produced. 



3.8.6 Bridging education 

Bridging education aims to teach learners “the requisite academic skills to a level 
sufficient to enable them to enrol in other tertiary programmes to which they would 
not otherwise be able to gain entry” (Benseman & Russ, 2003, p. 43). In New 
Zealand bridging education has developed in an ad hoc manner rather than as the 
result of government policy so it varies considerably between individual tertiary 
institutions. It is assumed that bridging education includes some elements of LNL 
teaching but little is known as yet about the nature and extent of that provision. 

We have located only one study on bridging education 106 effectiveness (Boylan & 
Bliss, 1997). This article is based on the on-going National Study of Developmental 
Education , which set out to assess the efficacy of developmental education across 
the US. The study first identified programme components associated with student 
success in these programmes from a literature review. The study then analysed data 
from 6,000 students who had completed courses at a random selection of 160 
institutions against these factors. This causal-comparative method enabled the study 
of relationships between multiple variables. The authors concluded that three 
characteristics identified from the literature were strongly associated (p>0.05) with 
student success in these programmes: 

> centralised programmes and especially their co-ordination (rather than where 
programmes are run in individual academic departments) 

> tutoring services that have tutor training 

> regular and systematic programme evaluation (although only 14% of 
programmes do this), leading to programme improvements (p. 6). 

The authors acknowledge that other factors are also associated with programme 
success, but these have only a weak association or were not found in sufficient 
numbers of programmes to assess. 



106 The term bridging education tends to be used in New Zealand, developmental education in the US 
and access education in Europe. 
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4 CONCLUSIONS 



The purpose of this literature review is to provide a critical evaluation of research 
evidence about effective practices in adult LNL teaching and programme provision, to 
inform the development of foundation learning policy and LNL teaching practice. This 
review has sought original research studies that relate specific aspects of teaching 
and programmes’ operations to learning outcomes - especially demonstrable 
changes in the literacy skills of learners. 



4.1 Findings 

The findings of this review need to be considered as tentative, due to the limitations 
of the research base from which they are drawn. The difficulties of researching adult 
LNL have already been described; research to date has been of variable quality and 
much of it has involved such small sample size that it is difficult to generalise from 
the findings. Therefore, we cannot present a list of factors that will definitively ensure 
effective teaching of adult literacy, numeracy and language learners and result in 
learners’ gaining LNL skills. 

However, an analysis across the strongest studies and reviews found some 
congruency of findings, which enables us to group those findings and have some 
confidence that the particular factors identified are likely to contribute to learner gain. 

Factors that appear likely to enhance learner gain include: 

> Appropriately skilled teachers who can identify the strengths and weaknesses 
learners have in speaking, reading, writing and numeracy.™ 7 Findings from 
two studies suggest that full-time teachers are more likely to enhance learner 
gain, and that learners benefit when there is assistance from teacher aides or 
volunteer tutors. Findings also suggest the importance of teachers having 
adequate non-teaching time for planning and professional development 
(Basic Skills Agency, 2000; Benseman, 2001; G Brooks et al., 2001; 
Fitzgerald & Young, 1997; Kruidenier, 2002b; Padak, Sapin et al., 2002). 

> Deliberate and sustained acts of teaching, clearly focused on learners’ 
diagnosed needs. All LNL learners, including those who have learning 
disabilities or dyslexia, would benefit from teachers who are able to offer a 
range of teaching strategies. Many studies commented that the actual amount 
of deliberate teaching in LNL programmes was often not very high. (Basic 
Skills Agency, 2000; Condelli, 2003; Condelli & Wrigley, 2004a; Kruidenier, 
2002b; Rice & Brooks, 2004; H.S Wrigley, 2003) 

> A curriculum that is linked to the authentic literacy events that learners 
experience in their lives. An authentic curriculum appears to lead to gain for 
learners in family literacy, for ESOL learners and for mainstream LNL learners 
(supported by more tentative findings that an authentic curriculum assists in 
achieving learner gain in workplace and prison programmes). (Condelli, 2003; 



107 This finding resonates with the research on effectiveness in New Zealand schools (Hattie, 2002a), 
where “what teachers know, do and care about” is recognised as one of the greatest influences on 
student learning (p. 7). 
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Condelli & Wrigley, 2004a; Kruidenier, 2002b; Padak & Padak, 1991; Purcell- 
Gates et al., 2002) 

> Programmes that allow for high levels of participation, probably more than 
100 hours of tuition. Learners with low levels of skill need more tuition for 
longer, as do ESOL learners. It should be noted that the research reviewed 
had a mix of findings. Three studies (including one ESOL study) found 
learners made gain when receiving over 100 hours teaching (with one study 
suggesting learners would need 300+ hours to move between levels). Two 
other findings suggested fewer hours might also be effective; one of these 
found learners improved when learners had more than 50 hours provision; 
while in another the greatest level of improvement occurred when learners 
had 51-60 hours between pre- and post-tests. Researchers commented that, 
regardless of the exact number of hours before learners made gains, for 
many LNL participants the total hours of teaching received were likely to be 
considerably less than the figures suggested above. While most of the 
research focused on total amount of provision, the intensity or regularity of 
tuition is probably also important. For example, one study suggested that 
learners made less gain once they received more than nine hours teaching 
per week. Intensive courses may be particularly important. For some ESOL 
learners, regular attendance was a more significant variable in achieving skills 
than the actual hours per week. (Basic Skills Agency, 2000; Boudett & 
Friedlander, 1997; Comings, 2003; Kruidenier, 2002b; Shameem et al., 2002) 

> Explicit teaching of reading, by teachers who are well trained in the reading 
process, and who are skilled in identifying reading difficulties and using 
appropriate teaching strategies to address them. Findings suggested learners 
are more likely to make gain when there is explicit, structured teaching of 
alphabetics, fluency, vocabulary building and comprehension. ESOL learners 
gain when there is explicit teaching of comprehension and vocabulary. 
Reciprocal reading was identified as an effective teaching strategy; others 
what warrant further investigation include oral reading, strategies to increase 
comprehension and fluency-related strategies. (Besser et al., 2004; 
Kruidenier, 2002b; Rich & Shepherd, 1993; Sabatini, 2002; Snow & Strucker, 
2000 ) 

> On-going assessment that takes into account the variation in learners’ skills 
across the dimensions of reading and writing. Assessment processes need to 
incorporate measurement of all four components of reading: alphabetics, 
fluency, vocabulary and comprehension. The assessment of reading needs to 
use more than silent reading and oral comprehension questions as 
assessment tools. A study of learners’ perspectives also identified the 
importance of on-going assessment. (Besser et al., 2004; Davidson & 
Strucker, 2002; Kruidenier, 2002b; Ward, 2003) 

> ESOL programmes structured to maximise oral communication, discussion 
and group work. Findings suggest learners make gain, particularly in 
comprehension, when there are bi-lingual tutors who teach concepts and 
explain learning tasks and instructions using learners’ first languages and 
when they attend regularly. (Condelli & Wrigley, 2004; Condelli & Wrigley, 
2004a; C. Roberts et al., 2004; Shameem et al., 2002) 

> Programmes that deliver clearly structured teaching using a range of 
methods. Provision needs to occur in a range of contexts that: meets 
learners’ needs; that allows for learning plans for every learner; and where 
those plans are related to regular assessments and reviews with learners. 
Most of these factors are not necessarily unique to LNL teaching - they 
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should be components of any quality adult education provision. (Basic Skills 
Agency, 2000; Boudett & Friedlander, 1997; G Brooks et al., 2001 ; G. Brooks 
et al., 2001; Comings et al., 1999, 2000; Condelli & Wrigley, 2004a; Eldred, 
2002; Yaffe & Williams, 1998) 

> Writing programmes that use writing based on expressing learners’ 
experiences and opinions. Programmes that include project-based instruction 
that focus on issues of common interest, on authentic tasks and materials and 
that encourage a variety of writing activities are more likely to promote gain. 
(Gillespie, 2001 ; Kelly et al., 2004; Purcell-Gates et al., 2002) 

> Making efforts to retain learners, including pro-active management of the 
positive and negative forces that help and hinder persistence. Findings 
suggest that childcare, transport assistance, and access to social services 
make a difference, as does building self-efficacy and self-confidence in 
learners and ensuring that learners receive personalised attention. (Comings 
et al., 1999; Eldred, 2002; B. A. Quigley & Uhland, 2000; Yaffe & Williams, 
1998) 

> Family literacy programmes that have parents committed to improving their 
children’s chances. Programmes that have an impact have a clear focus on 
literacy/numeracy development; have teaching sessions for parents only and 
children only, as well as together; have home visits; collaborate with other 
participating groups, to ensure programme and services integration; and have 
staff whose skills match the unique challenges of family literacy. (Alamprese, 
2001b; Benseman, 2002, 2003c, 2004; Padak, Rasinski et al., 2002; Philliber 
et al., 1996; St Pierre et al., 2003; St. Pierre et al., 1995) 



There are some factors that may enhance learner gain, but for which there has been 
only limited research and where findings are even more tentative: 

> Programmes that have an awareness and management of critical periods in 
the programme (when learners were susceptible to failure and/or withdrawal); 
provide pastoral care in times of need and crisis; have adequate fiscal 
resources and facilities; have administrative leadership. 

> Teaching staff who undertake regular professional development; praise and 
encourage learners; are open as people to their students and have both 
credibility in their field and a passion for their subject. 

> Assessment that includes self-assessment by learners, and constructive, 
verbal feedback from the tutor. 

> Teaching that creates a positive and supportive learning environment; is able 
to balance challenge and support for learners; encourages peer support in the 
form of sharing skills and ideas and friendships; accentuates learners’ 
strengths. Again, these attributes appear to be aspects of good adult 
education practice. 

> Computer Assisted Instruction may be more effective in teaching 
mathematics and other aspects of LNL in comparison with ‘traditional’ 
teaching and with learners with low levels of skill; however, it does not replace 
skilled teachers; it may motivate learners and develop self-confidence. 

> Reading programmes that ensure learners’ prior knowledge is used from the 
beginning rather than assuming that adult beginning readers need to 
concentrate on strategies for processing print first; that include various 
strategies related to teaching alphabetics, fluency, vocabulary and 
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comprehension as suggested in K-12 research; that teach learners to monitor 
their comprehension and understanding as they read; and that are at least 
70% on-task (i.e. that the majority of time is spent on reading and writing 
tasks). 

> Family literacy programmes that: have public celebrations of key events and 
achievements; and that have a ‘second step’ programme for the adult 
learners as they move on from the programme to maintain the “community of 
learning”. 

> Community-based programmes that: have community ownership and 
resources; and where there is trust and collaboration between providers and 
communities. 

> Bridging programmes that: are centralised, especially their co-ordination 
(rather than being run in individual academic departments). 

> Workplace literacy programmes where there is commitment from the 
participating company, where tuition occurs during work time and where 
curriculum is related to real-life demands. 

> Prison-based programmes that: incorporate a community-building process 
into their reading programme; and where there is contextualised content in 
reading teaching, as this increases attendance rates and levels of 
engagement. 



Fourthly, there are factors that are not supported by research evidence at this time. 

> There are no clear findings that one form of delivery (1 :1 /small group/class) or 
context is better than any other. There were a number of studies with very 
small numbers of learners saying 1 :1 is effective, as well as others saying that 
participation in small groups is what makes programmes ‘not like school’. 
There was evidence that a range of factors impact on effectiveness in any 
form of delivery or context including, for example: students’ motivations to 
attend and their life stages, the language and literacy diversity of students 
within a group, strategies used to retain students, the extent programmes use 
authentic contexts and materials, the intensity of instruction and the skill of 
the teachers. 

> The research evidence on numeracy does not provide guidance on the time 
and the nature and extent of teaching required for adults to make significant 
progress. 

> There is no evidence to support a policy of differentiating dyslexic from non- 
dyslexic students in adult literacy, numeracy and ESOL. 



Finally, we were not able to identify any research that met the criteria for our study on 
the following subjects listed in the research brief. 

> integrated or embedded provision 

> any socio-demographic characteristics of tutors 

> strategies related to teaching te reo Maori or Maori programme provision 

> factors associated with progress in numeracy 

> assessment and its effect on learning outcomes. 
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4.2 Developing a cycle of research that informs practice 
and policy 

This review has illustrated the limitations of the available LNL research, limitations 
reiterated in all of the international literature reviews we have encountered for this 
project (Beder, 1999; G Brooks et al., 2001; G. Brooks et al., 2001; Campbell, 2003; 
Condelli & Wrigley, 2004; Torgerson et al., 2004; H.S. Wrigley, 2003). We already 
know that we lack quality LNL research specific to New Zealand (Benseman, 2003a), 
especially research that can provide useful findings to inform future policy directions 
and the development of good quality provision. 

We have to improve the quality of the research that is undertaken - again, this has 
been an issue worldwide. Beder (1999) has described the limitations in American 
research completed prior to 1999, including for example: inaccurate or incomplete 
data; over-reliance on self-report data; lack of adequate controls; lack of valid, 
reliable and appropriate tests; poor quality research reports and a lack of relevant 
standards. His review identified how the nature of the environment in which 
foundation learning takes place makes research particularly difficult (factors that are 
also found in New Zealand) - open enrolment, high dropout rates, intrusion of events 
and crises arising from adults’ daily lives and responsibilities and sporadic 
attendance. The lack of capacity in local programmes to collect accurate and timely 
data to monitor provision is another issue, given the low budgets, high staff turnover 
and reliance on part-time staffing in many programmes (again, all factors common in 
the sector in New Zealand). 

In his paper on establishing an evidence-based system, Comings (2003, p. 9) 
proposes a process that follows a set of recurring cycle of steps in order to improve 
the links between research and practice: 

> a review of existing research and professional wisdom to inform the design of 
baseline models for teaching and support services that conform to the best 
available evidence 

> evaluation of these baseline models to establish outcomes and impact 

> practitioners using the results to inform and make decisions about their 
practice 

> practitioners’ experiences of putting the models into practice are shared 

> based on this review, additional research is undertaken, thereby leading to a 
constant revision of the model(s). 



He argues that this process can not only lead to refining existing models of practice, 
but also allows new alternative models to be explored and provides a sharper focus 
for professional development. 

Beder (op cit) argues that successful research in future will require budgets adequate 
for the size and complexity of the task, multi-skilled research teams, including 
technical experts skilled in research (both quantitative and qualitative) and 
experienced researchers and practitioners who understand the particular nature of 
the adult literacy context. 

Future LNL research for adults in New Zealand therefore needs to take into account 
a number of factors that have emerged in undertaking this literature review: 
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increasing the availability of the research that is being done, keeping in close touch 
with overseas research programmes; maximising the usefulness of local research 
and ensuring ready incorporation of research into professional practice. 



4.2.1 Availability of existing research 

In the process of undertaking this review, we found reference to a number of studies 
that have been undertaken in New Zealand, but that have not been made publicly 
available for a range of reasons. If we are to develop an evidence-based system of 
LNL for adults, it is essential that funding agencies release these studies into the 
public arena, even when they have weaknesses, in order to stimulate professional 
debate, not only about the LNL elements concerned, but also the research 
methodologies and processes involved. Ready access to research is an important 
component of encouraging debate and discussion between researchers, policy- 
makers and practitioners; and the present paucity of research (together with the 
smallness of the field generally, which will limit the research undertaken) makes it 
imperative that what is carried out is readily available in the public domain. 



4.2.2 Tracking overseas LNL research 

A considerable number of substantial, intervention studies are currently underway 
overseas that are highly relevant to the aims of this review. Most of this new 
generation of studies is related to the national research programmes of the National 
Research and Development Centre (NRDC) in Britain ( www.nrdc.orq.uk) or the 
National Center for Study of Adult Learning and Literacy (NCSALL) in the United 
States (http://www.qse.harvard.edu/~ncsall/) . Unlike much of the research discussed in 
this report, the current studies are often the result of national, co-ordinated research 
programmes run by collaborations of organisations rather than individual 
researchers’ studies or localised evaluations. The results of these studies are likely to 
be very useful to us in the future. Examples of studies underway include: 

> NCSALL is studying the effect of sustained silent reading, based at the 
National Labsite for Adult ESOL; a report is due on 2005. 

> NCSALL’s National Labsite for Adult Literacy Education is undertaking a 
range of projects including: 

o identifying the factors that engage and/or disengage learners and the 
relationships between engagement and dropout 
o identifying and testing best practices for managing mixed level/ 
continuous enrolment classes 

o studying the relationships between classroom instruction and literacy 
acquisition in learners’ homes, workplaces, and communities 
o investigating the relationship between teachers’ beliefs about how 
instruction should be conducted and how teachers actually teach 
o a longitudinal outcome study that measures learner progress over 
time 

> A major experimental study, Building a Knowledge Base for Teaching Adult 
Decoding involving a structured reading intervention that focused on decoding 
and fluency, is being tested over three years in the USA, involving 
approximately 23 ABE programmes in 12 states. The project is being funded 
by a collaboration of US government departments and literacy organisations. 
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Information about the programme can be found on 
http://www.abtassociates.com/presentations/Proi ADULTREAD 5-04-04. ppt 

> An NR DC project Effective approaches to the teaching and learning of writing 
is due to be reported on in 2005. The project aims to provide an overview of 
what is known about the development of writing skills in adults up to level two. 
It will investigate the correlations between classroom practice and the 
progress of learners in the development of the skills of writing and their 
confidence as writers. 

> An NRDC study entitled Numeracy: Effective Teaching and Learning is due 
for completion in 2006, exploring the correlation between different 
pedagogical practices and learners’ progress in a range of settings. 

> Another NRDC study is considering initiatives intended to improve the 
literacy, language and numeracy skills of adults through workplace-linked 
tuition. Adadiadou et al (2004) details the methodology they have developed 
and their plans to sample 400 workplace LNL learners. The project is due for 
completion in 2005. 

> An in-depth set of case studies of embedded LNL research projects is 
underway in the UK and is due to be reported on late during 2005, which may 
give more guidance on how best to provide effective integrated teaching. 
Information may be obtained from NRDC or NIACE ( www.niace.org.uk) . 

> The National Council for Vocational Education Research (NCVER) in 
Australia ( www.ncver.edu.au) has also recently commissioned case study 
research, including observations of what teachers actually do in the Australian 
context that will be useful to consider. 



Many of the studies above have preliminary publications available that discuss 
methodologies and issues that have arisen in the scoping of these projects. 
Researchers in New Zealand would benefit from considering these as part of 
preparations for undertaking research here. 



4.2.3 Maximising usefulness of local research 

It is important to stress that we are still unable to state that particular LNL strategies 
and techniques do, or do not, lead to learner gain with the degree of confidence that 
is possible in many aspects of schooling. Rather, the ‘jury is still out’ - there is not a 
sufficiently large research base as yet from which to draw strong conclusions, even 
though this situation is clearly undergoing substantial change at present. In two-three 
years time, a similar review should be able to produce more useful research findings. 

As funding arrangements, provision and practice in foundation learning evolve in 
New Zealand over the next few years and new research projects come on stream, in 
keeping with Comings’ model we need to look more closely at the quality of the 
methodologies. While we are unlikely to ever replicate the size of large international 
studies, we do need to look at qualities of overseas studies to see what we can 
usefully replicate and expand on in contexts specific to New Zealand. 

For example, there is a need to move beyond exploratory, small-scale studies with 
limited sample size to research with sampling frames of sufficient size and 
methodologies of substantive enough quality that findings can be more readily 
generalised than has been the case to date. It may be more useful to limit the overall 
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number of studies being undertaken and broaden their insights and degree of 
applicability beyond specific contexts. 

It is essential that all research builds on what has been carried out previously - in this 
way, researchers contribute to the accumulated wisdom of other researchers (even if 
it is to avoid the same errors) by ‘standing on their shoulders’. It is essential therefore 
that research funders explicitly stipulate that all projects include a literature review 
and its findings are fully integrated into the main study. 

The number of LNL researchers in New Zealand is very small. There would be 
benefits in collaborating with LNL researchers internationally to grow our pool of 
expertise and to look for expertise from other components of education to enhance 
the technical quality of our methodologies. 



4.2.4 Utilising research in provision 

Research counts for very little unless it ultimately contributes to improved 
professional practice. There is an urgent need to consider and plan ways to 
disseminate research findings, to encourage practitioners to review their teaching 
against these findings and to then modify their practices as Comings (2003) 
advocates. Simply publishing research is not enough. It is particularly important that 
research findings be incorporated into the planning of both initial training and on- 
going professional development for practitioners. 

There are no refereed foundation learning-related journals in New Zealand at 
present, which means it is difficult for studies to get peer review through the 
traditional academic process. 108 This difficulty is exacerbated by the lack of an 
overarching body for the foundation-learning sector, which makes it difficult to 
achieve ready dissemination of research findings. 

While the peer review process is valuable in helping to improve the quality of 
research, journals are seldom the most appropriate mechanism for ensuring that 
findings are read and utilised among practitioners, because very few practitioners will 
be in a position to access them. To be effective, dissemination of research findings 
has to be actively managed. It is interesting to note for example, that the NCVER in 
Australia tags a considerable proportion of its budget to active dissemination of their 
research studies’ findings using a range of strategies for this purpose. NRDC run a 
programme of seminars to disseminate research findings after each study. 

In New Zealand the Literacy Portal has potential to further develop its dissemination 
role in relation to research. For example, the large number of research reports 
identified and sourced for this study (both electronic and hard copy) could be 
publicised, promoted and disseminated using the Literacy Portal as well as offering 
full copies of completed research reports. At present there is no repository for the 
paper-based material and no systematic means of cataloguing any of it to make it 
available for other researchers. 



108 The New Zealand Journal of Adult Learning is the only local refereed publication that has published 
any number of studies related to adult LNL, although its focus is on adult learning generally rather than 
LNL. Other, more ABE practitioner-focused publications, such as Literacy Works by Workbase could 
possibly be extended for this function. Australian publications such as Australasian Journal of Vocational 
Education and Fine Print may be more appropriate, but are probably read by a limited number of New 
Zealand-based readers at present. 
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Immediate consideration should be given to the dissemination of this review and all 
other LNL-related research studies currently being completed. 



4.3 Recommendations 

In keeping with these general comments about future LNL research directions in New 
Zealand, we believe that a number of studies should be undertaken over the medium 
term. This list does not constitute a strategic research programme, but rather draws 
on key findings in the research in order to productively focus some future research. 

1. LNL teachers: this review has showed how central LNL teachers are in 
achieving learner outcomes and yet we know remarkably little about who 
these people are, the contexts and nature of their employment and the 
organisation of their work, their motivations and aspirations, training, 
issues and philosophies of teaching and the impact of professional 
development they undertake etc. A large-scale survey would provide 
useful baseline information for the future expansion of both initial and on- 
going professional development programmes. Any study should 
encompass whether tutors work with ESOL learners and/or teach 
numeracy. When investigating numeracy information would be useful on 
what they teach, what their professional development needs are; and their 
beliefs and perceptions of self-efficacy related to their own numeracy. 

2. Specific teaching methods for reading: New Zealand has had a long 
history of research into child-related reading that has never been tapped 
into by the LNL sector. There is scope to take some of the methods 
identified as potentially useful in this review and construct research to test 
their effectiveness in New Zealand LNL contexts. Some aspects that 
warrant attention include: reciprocal reading, which the research indicates 
is an effective means for teaching adults, oral reading, and strategies to 
develop fluency and comprehension. 

3. Authentic curriculum: it would be very useful to identify to what extent 
authentic curriculum is being incorporated into teaching across the variety 
of contexts of LNL provision in New Zealand, and analyse issues around 
its use and the opportunities this approach offers us. 

4. Learner-focused longitudinal research: a longitudinal, intensive study 
of approximately 20-30 learners over two years (with the expectation of 5- 
10 dropping out over that period) could provide information about what 
learners believe they learn, how their LNL skills progress and what they 
believe has helped and hindered them. A case study approach would also 
provide a rich description of additional programme factors, for example, 
effective recruitment, attendance patterns, barriers to participation, and 
learner progression on to other outcomes in work and education. 
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APPENDIX A - PROFESSOR J OHN HATTIE'S ANALYSIS OF 
STUDIES IN TORGERSON ET AL. (2004) 



Torgerson et al. (2004) identify 26 studies from a total of 4,555 potentially relevant 
papers reporting evaluations of interventions in adult literacy and/or numeracy, and 
published between 1980 and 2002. Only those studies that used randomised 
controlled trials (RCTs) and controlled trials (CTs) were selected, hence only 29 
studies were included in the final report. We were able to locate 21 of these studies, 
which is sufficient to demonstrate the difficulties of reviewing by choosing only 
studies that met a certain set of design specifications. 

It is noted that a major contribution of meta-analysis is to include all possible studies 
and then ask the question whether the nature of the design (and many other features 
of the study) makes a difference. It may be that the nature of the design is not related 
to the size of the effects and thus a stronger conclusion can be made about more 
critical and educational attributes. To decide on only including certain types of 
designs, before such an analysis, presupposes that the studies using only the 
specified designs are best representative of the population estimates. This is 
speculation, and using meta-analysis, subject to verification. 

Moreover, choosing specified designs, no matter how defensible, assumes that 
studies using these designs are necessarily somehow superior to studies using other 
designs. What is striking about the 21 studies we located from Torgerson et al. is 
their variable quality - with most of low quality. Using randomised or controlled 
designs clearly does not lead to high quality studies. The median sample size is only 
52, and given there are at least two groups (experimental and control) the “typical” 
study has only 26 people in each group. The average attrition rate is 66%, so two- 
thirds of each sample did not complete the study. It would have been more 
defensible to include all possible studies, code them for the nature of the 
experimental design, and for the quality of the study and then use meta-analysis 
techniques to address whether the effects differ as a consequence of design and 
quality. The aim should be to summarise high quality studies, regardless of their 
design - if quality is a moderator to the final conclusions. 

Randomised design 

The usual design is a pre-post test with experimental and control groups, but for 
many studies it is very evident that the randomisation into these two groups failed to 
ensure pre-test comparability. One of the major purposes of randomisation is to 
“equate” the experimental and control groups - but it is noted that randomisation is a 
“method” aimed to ensure that the two groups are not too dissimilar, it does not 
guarantee it. The ultimate criterion is “no difference” in the pre-tests, and 
randomisation is one way that may lead to this criterion. In one-third of the studies we 
located it was most apparent that this was not the case: the pre-test means for the 
experimental and control groups were markedly different, indicating that random 
assignment did not lead to “representativeness” (Dietrich, 1994; Dirkx & Crawford, 
1993; Lavery et al, 1998; MacLay & Askov, 1988; Nicol & Anderson, 2000; Shrum, 
1985; Thuy, 1992). 

For example, Dietrich (1994) “randomly assigned” each individual to one of two 
instructional conditions - a traditional meta-cognitive approach, and the experimental 
phonological skills approach. Despite such random assignment, there are major 
differences in the pre-test means: the auditory concept (ES=.37) word attack skills 
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(ES=1 .60), and word identification means (ES=.13) were lower in the control groups, 
questioning the power of random assignment. The sample size in each group is only 
about 10, and even then one missing score in the control group was “replaced by the 
mean for that student.” Further, despite such attention to randomisation, the author 
stated that “my instincts as a teacher interfered with my knowledge as a researcher 
of the need to achieve randomness in my groups. Instinctively, I encouraged those 
students most in need of phonological help to participate in the experimental group” 

(p. 12). 

Thuy (1992) assigned adults into two groups: an experimental group that received a 
CAI course and a control group which received the same course without the 
computers. It is clear from the means (see below, standard deviations were not 
provided) that there were substantial pre-test differences even between the control 
and the experimental group with the lower beginning scores. It is difficult to imagine 
this was possible if random assignment was used. 



Table 1. Pre-test means for Experimental and Control groups on 
Vocabulary and Comprehension after “randomisation” for Thuy (1992) 





Vocabulary 


Comprehension 


Reading measures 


Experimental 


Control 


Experimental 


Control 


Paper Bag Princess 


3.19 


3.13 


46.00 


9.13 


Thomas' Snowsuit 


16.23 


3.00 


20.33 


12.17 


Classified Ads 


3.40 


5.20 


.00 


2.87 


Christmas Season 


11.71 


8.42 


1.86 


3.83 



MacLay and Askov (1988) did not use randomisation, but used a convenience 
sample of those “interested in and eligible for using the courseware but because of 
scheduling problems, transportation, and child care were unable to participate at this 
time” (p. 24). Further, the effect-sizes reported in Torgerson et al. for the pre-test 
means across the two groups are in some cases considerable (.26, .24, .73, .16), 
questioning the comparability of these groups. Thus, the post-test means between 
the control and experimental groups (.63, 1.31, 1.55, 1.02) suggests that these pre- 
test differences should, at least, be subtracted to ensure some comparability. In our 
meta-analysis we used the difference between the experimental and control at post- 
test minus the pre-test differences (.37, 1 .06, .81 , .85). 

Askov et al. (1986) used 12 students in a time-series design, and then compared 
these students’ pre-and post-test gains with a group of 27 inmates. There is no 
evidence that the prisoners were assigned randomly to the experimental and control 
groups. They seemed a convenience sample and it is difficult to see how it could be 
classified a “Controlled design”. Certainly the time series part of the study could have 
been so designated, but it was not this part that is used in Torgerson et al. Further, 
the pre-test mean for the Experimental group was .08 and for the Control group .40 - 
hardly comparable groups. If it is reasonable to assume a constant sd for the pre- 
and post- means for the control and experimental groups of about a quarter of the 
means (given a normal distribution of ability), then the effect-size difference of the 
experimental and control groups at pre-tests is 5.33, and at post-test is 4.07, showing 
a gain of 1 .27. 

These studies illustrate that a randomised or controlled design does not necessarily 
equate with high quality and, at minimum, Torgerson et al. should have coded for the 
quality of the study and allowed for this moderator in their interpretations and 
conclusions. 
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Effect-sizes 

Torgerson et al. (2004) calculated the effect-sizes for each study, and it is clear that 
in many instances they did not attend to any pre-test differences. It seems that they 
calculated an effect-size as the difference between post-test means of the 
experimental and control groups divided by the pooled standard deviation. The 
correct effect-sizes should have been based on an analysis of covariance with pre- 
test scores as the covariate (but such data were not available to Torgenson et al.). It 
seems important to allow for pre-test differences between the groups and thus, 
where available, we used the differences between the experimental-control groups 
post-test differences and subtracted the differences from the pre-test differences 
([Expp OS ,-Cntpost] -[Exppre-Cntpre]). We should have also allowed for a correction due to 
their being a positive correlation over time in the dependent variable. This has the 
effect of reducing the standard deviations by (1-r 2 ), where r is typically about .8 for 
achievement tests. 

Notwithstanding, there are discrepancies between the effect-sizes we calculated from 
many of the articles and those reported in the Report. In most cases the differences 
is because of the above noted effect-size formulae. We cannot, however, ascertain 
how Torgerson et al. (2004) arrived at their estimates of effect-sizes for Dietrich 
(1994) even allowing for these different formulae (they have .85, -.63, ad -1.03 for 
LAC, Word attack, and Word identification, respectively whereas we calculate 1.34, 
.70, and .49). In Lavery et al. (1998) Torgerson claimed that the effect-size for the 
Burt, for example, is -1.24, and that “contrary to author’s claims” there were no 
differences. We find it difficult to see how they can make this claim: the pre- and 
post-means for the control groups are 95.33 and 98.17, and for the experimental 
groups are 50.00 and 80.17; certainly the effect-size cannot be negative, and our 
estimated effects of .29 for pre and 1 .44 for post are quite substantial. Similarly we 
calculated .43 compared to .70 for Shrum (1985); .24 vs. .77 for Bean and Wilson 
(1989), and .71 vs. .50 for Nicol and Anderson (2000). The latter two are probably 
because Torgerson et al. did not consider the pre-test differences. We could not 
replicate the effect-size of 23.93 for Roberts et al. (1994) for the “Gain mean”. 

Results 

The following table presents the mean effect-size per study from the Torgerson et al. 
(2004) report, from our calculations, the best estimate from either, and comments 
where there are major discrepancies. It is noted that one study, Thuy (1992) had an 
extremely high effect-size (6.96). As noted above, this is primarily because of major 
differences in the pre-test means between the experimental and control groups but 
we were unable to locate this study to make any adjustments. 
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Table 2. Mean effect-size calculations from Torgeson et al (2004) 



Table 3. 



Author 


Design 


Report 


Ours 


Best estimate 


Comments 


Askov et al. 


CT 




1.26 


1.26 




Bean & Wilson 


RCT 


.77 


.24 


.24 


Report ignores pre- 
test differences 


Batchelder & Rachal 


RCT 


.29 


.17 


.17 




Broughton 


CT 




.20 


.20 




Broussard 


CT 










Burtoff 


CT 










Cheek & Lindsey 


RCT 


.46 


.44 


.44 




Culclasure 


CT 










Diem & Fairweather 


CT 


.01 




.01 




Dietrich 


CT 


-.27 


.84 


.84 


Cannot be -vs 
when means higher 
for Exp at post 


Dirkx & Crawford 


CT 




.53 


.53 




Gretes & Green 


CT 


.97 


.98 


.97 




Indiana 


CT 










Irby et al 


CT 










Lavery 


CT 


-.08 


.86 


.86 


Report ignores pre- 
test differences 


Lehigh 


CT 




-.36 


-.36 




MacLay & Askov 


CT 


1.13 


.77 


.95 




Macmurdo 


CT 










Martinson& Friedlander 












McKane & Greene 


RCT 


.20 


.16 


.16 




Morrow 


CT 




.66 


.66 




Myer 


CT 


.22 




.23 




Nicol & Anderson 


RCT 


.50 


.71 


.71 


Report ignores pre- 
test differences 


Nurss 


CT 


-1.65 




-1.65 




Rich & Shepherd 


RCT 


1.14 


.86 


.86 




Rio Salade 


CT 




.40 


.40 




Roberts 


CT 


1.06 


.38 


.38 




Schrader 


CT 










Shrum 


RCT 


.71 


.43 


.57 




Smith & Dalheim 


CT 










St Pierre et al. 


RCT 


.29 


.30 


.30 




Thuy 


CT 




6.96 


6.96 




Wadsworth & Frasier 


CT 










Washington 


CT 










Wilson 


CT 




.48 


.48 




Wisher & O'Hara 


CT 
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The average effect-size from the Torgerson et al. (2004) report is .35 (sd = .69), from 
our calculations .82 (sd=1.45). When the Thuy study is omitted (as it needs to be), 
then the means are .35 (sd=.69) from Torgerson et al., .51 (sd=36) from our 
calculations, and the best estimate is .40 (sd=.38). The best estimate of effect size is 
between .40 and .51, which indicates that an adult undertaking a program to enhance 
literacy outcomes increases their proficiency by about .4 to .5 a standard deviation. 
Hattie (1992, 1999) has estimated the typical effect-size from about 300,000+ studies 
covering many interventions (school, teacher, student, home, etc.) aimed at school- 
aged students to be .40. Thus, relative to other educational interventions, the effects 
of adult literacy programs can be classified as an “average” intervention. 

The following stem-and leaf diagram presents all effect-sizes, and it can be seen that 
they are centred around .3 to .8, there is a positive skew, and there are two outliers 
(which if removed, make little difference to the messages although dramatically 
reduce the variance). 



Effect 


No. 


-1.2 


9 


-0.3 

-0.2 


4 


-0.1 


7 


0 


3 


0 


25 


0.1 


2225577 


0.2 


1 


0.3 


4566799 


0.4 


3359 


0.5 


23336679 


0.6 


2 


0.7 


359 


0.8 


1556 


0.9 


28 


1 


6 


1.1 


45 


1.2 


6 


1.3 


4 


1.7 


3 


6.9 


6 



The effect-sizes from journals, where there is a degree of peer quality review (.57, 
No. = 32) and from theses (.20, No. =4) were much lower than from Reports (.74, No. 
= 19). Similarly, the means from the randomised control trials (.43, No. =20) are lower 
than for Controlled trials (.71, No. = 35). The overall mean is more likely to be at the 
lower end of the best estimates of .40 to .51 . 

The effects were higher in Reading (.67, No. =45) than in Numeracy (.31, No. = 9). 
Adults in the programs had a somewhat higher effect (.68, No. 35) than younger 
adults (16-25 years, .48, No. = 7), or in a prison (.49, N=13). The correlation between 
the number of participants and the effect-size was zero (-.06, No. = 55), but very high 
with retention (.88) indicating that the programs with the highest retention rates had 
the greatest overall effects. There was a low correlation between the number of 
hours of the intervention and the effect-size (-.06, No. = 55). 
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Concluding comments 

This collection of articles clearly points to the delimitations in making a method of 
assignment (randomisation) the major consideration in deciding on the quality of 
studies. It is difficult to imagine a more unconvincing collection of studies - with low 
sample size, high attrition, low statistical power, and so many pre-test differences. As 
well, there are so few studies that critical moderators cannot be investigated in any 
meaningfully generalisable way to evaluate their potential influences on the effect- 
sizes. Most important, if more studies had been included then the key assumption, 
that the effect-sizes from RCT and CT are different according to the design of the 
study, could have been evaluated. 

As Scriven (2004) has argued a more critical criterion for all scientific conclusions is 
“beyond reasonable doubt (BRD)”, and in some cases randomised studies do not 
come close to BRD. “It seems more appropriate to think of ‘gold standard’ designs in 
causal research as those that meet the BRD standard, rather than those that have 
certain design features. ... The existence of more threats to internal or external 
validity in quasi-experimental designs does not entail a reduction of validity for well- 
done studies below BRD levels”. Scriven notes one of the advocates, Cook who 
claimed that “Interpreting [RCT’s] results depends on many other things — an 
unbiased assignment process, adequate statistical power, a consent process that 
does not distort the populations to which results can be generalized, and the absence 
of treatment-correlated attrition, resentful demoralization, treatment seepage and 
other unintended products of comparing treatments. Dealing with these matters 
requires observation, analysis and argumentation.” (Cook, 2004) The last sentence 
opens up the possibility that so many other research designs may deal with these 
issues. 

Perhaps the most marked criticism of the Torgerson et al. review is that they provide 
few implications for adult literacy - other than comments about research design. 
Undertaking a review, such as a meta-analysis, is fundamentally about inferences, 
implications, and conclusions about the substantive topic - adult literacy. They 
conclude “it is difficult to make any recommendations as to the type of adult 
education that should be supported” (p. 12)1 Thence, they ask for more RCTs - 
surely not of the quality so far produced. 

It is recommended that either a more substantial number of quality articles be 
included in a meta-analysis or a substantial review of adult literacy programs as then 
the more important moderators could be evaluated. 

John Hattie 
June 2004 
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APPENDIX B - LIST OF DATABASES AND WEB-PAGES 

SEARCHED 



ERIC 

http://www.eric.edu 

Expanded Academic 
Education Complete 
Psych Info 

Psych Lit and Behavioural Sciences 
Dissertation Abstracts 
ProQuest Social Sciences 

Literacy Research Centre - University of Sheffield - School of Education 

http://www.shef.ac.uk/literacv/ 

National Research and Development Centre for Adult Literacy and Numeracy 

http://www.nrdc.org.uk 

Lancaster Literacy Research Centre 

http://www.literacy.lancs.ac.uk/ 

Research and Practice in Adult Literacy Group (RaPAL) 

http://www.literacy.lancs.ac.uk/links/rapal.htm 

NIACE 

http://www.niace.org.uk 

Basic Skills 

http://www.basic-skills.co.uk/site/page.php?cms=Q 

European Basic skills network 

http://www.eurobasicskills.org/default.asp?site=1&lng=1&rgn=0&cat=0 

National Adult Literacy Agency(lreland) 

http://www.nala.ie/search/index.tmpl 

Community Learning Scotland 

http://www.communitvlearning.org/ 

Communities Scotland 

http://www.communitiesscotland.gov.uk/Web/Site/cl/al main.asp 

The Danish Research and Development Centre for Adult Education 

http://www.cvustork.dk/ufv.asp 

VOCB (Flemish Support and Development Centre for Adult Basic Education) 

http://www.vocb.be/vocb eng.html 

Latvian Adult Education Association (LAEA) 

http://home.parks.lv/laea/en/default.htm 

Federacion de Asociaciones de Educacion de personas Adultas (FAEA) 

http://www.faea.net/ 

OECD 

http://www.oecd.org 

National Centre for Study of Literacy and Learning 

http://gseweb.harvard.edu/~ncsall/index.html 

National Centre for ESL Literacy Education 

http://www.cal.org/ncle/ 

National Lab-site for Adult Literacy Education - Rutgers University 

http://ncsall-ru.gse.rutgers.edu/research.html 
New Zealand Literacy Portal 
http://www.nzliteracyportal.org.nz/NZ+Research/ 

Bureau of Adult Basic and Literacy Education 

http://www.able.state.pa.us/able/site/default.asp 
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Pennsylvania Action Research Network - Learning from practice 

http://www.learninqfrompractice.org/paarn/default.htm 

Pennsylvania Adult Literacy practitioner Inquiry network (PALPN) 

http://www.learninqfrompractice.orq/palpin/default.htm 

National Institute for Literacy 

http://www.nifl.qov/nifl/publications.html 

National Center for Family Literacy 

http://famlit.org/ 

Literacy Exchange: World Resources on Literacy 

http://www.literacvexchanqe.net/ 

UNESCO Institute for Education 

http://www.unesco.org/education/uie/publications/index.html 

Kluwer Academic Publishers 

http://www.kluweronline.com/issn/002Q-8566 
International Review of Education 

Literacy in Africa 
http://portal.unesco.org/education/ 

Centre for Literacy of Quebec 

http://www.nald.ca/litcent.htm 

ACTCAL -ACT Council of Adult Literacy 

http://www.acal.edu.au/actcal/index.htm 

Literacy and Numeracy Studies 

http://www.education.uts.edu.au/lns/about.html 

Alpha Plus Centre 

http://www.alphaplus.ca/enq.asp 

Professional Complete Collection database 

Psychology and Behavioral Sciences Collection database 

Centre for Language and Literacy - University of Technology Sydney 

http://www.education.uts.edu.au/centres/cll/publications/index.html 

JSTOR database 

Web of Science 

Sociological Abstracts 

King's College London 

http://www.kcl.ac.uk/depsta/education/ 

Adult Education Research conference 

http://www.edst.educ.ubc.ca/aerc/proceed.htm 

Canadian Association for the Study of Adult Education (CASAE) 

http://www.oise.utoronto.ca/CASAE/mainenq.html 

European Society for Research on the Education of Adults 

http://www.esrea.org/ 

Australian Education Index Database 

Digital Dissertations 

Google 

US Dept of Education 

Office of Vocational and Adult Education 

http://www.ed.qov/about/offices/list/ovae/pi/AdultEd/index.html 

National Centre for Educational Research 

http://www.ed.qov/about/offices/list/ies/ncer/index.html 

Comprehensive Adult Student Assessment Scheme (CASA) 

http://www.casas.org/ 

Division of Adult Education and Literacy (DAEL 

http://www.ed.gov/ 

National Assessment of Adult Literacy (NAAL) 

http://nces.ed.gov/naal/resources/resources.asp 
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Centre for Literacy Studies - University of Tennessee 

http://cls.coe.utk.edu/ 

Centre for Adult Learning and Literacy - University of Maine 

http://www.umaine.edu/call/pubs/pubs.html 

Center for the Study of Adult Literacy - University of Georgia 

http://education.qsu.edu/csal/site/pubs.htm 

National Center for Study of Adult Learning & Literacy - at Rutgers University 

http://ncsall-ru.qse.rutqers.edu/research.html 

DEW Research Group - Portland State University 

http://www.cedu.niu.edu/~smith/Adultlitpaqe.htm 
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APPENDIX C - DESCRIPTORS USED IN LITERATURE 

SEARCH 



1 . Adult literacy OR adult reading OR adult reading programmes OR literacy 
education OR illiteracy (sic) OR adult basic education OR adult basic 
skills OR basic skills 

2. Reading instruction OR reading skills 

3. Adult numeracy OR numeracy skills OR remedial maths 

4. Writing skills 

5. Oracy OR oral language OR second language instruction OR English skills 
OR ESOL OR TESOL 

6. Workplace literacy OR worksite education 

7. Correctional education OR prison education 

8. Impact analysis OR outcomes of education OR educational outcomes OR 
instructional outcomes OR learner outcomes OR results of education OR 
student outcomes OR 

9. Programme effectiveness OR teacher effectiveness OR instructional 
effectiveness 

10. Programme evaluation OR instructional evaluation OR pre-tests OR post 
tests 

1 1 . Teaching OR training OR instruction 

12. Computer-assisted instruction OR educational technology 

13. Teaching techniques OR teaching methods OR teaching practices OR 
educational methods 

14. Teacher characteristics OR educational environment 

15. Teachers OR tutors OR adult educators 

16. Teacher effectiveness OR teacher characteristics OR teaching conditions 

17. Curriculum OR programme design OR delivery systems 
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